diff --git a/README.md b/README.md
index 58e3170..b6436f9 100644
--- a/README.md
+++ b/README.md
@@ -4,17 +4,20 @@ A lightweight, modern web application built with Python (Flask) that streamlines
## 🚀 Features
-- **Direct Odoo Integration:** Seamlessly creates `res.partner` and `loyalty.card` records in Odoo 19.
-- **Data Mapping Automation:** Automatically maps database fields (including custom fields like `gender`, `birth_date`, and `total_spend`) and dynamically links the appropriate `loyalty.program` based on the user's membership level.
+- **Direct Odoo Integration:** Seamlessly creates or updates `res.partner` and `loyalty.card` records in Odoo 19.
+- **Smart Duplicate Prevention:** Automatically normalizes phone numbers (stripping non-digits and `0` or `62` prefixes) to find existing customers in Odoo. If a customer already exists, it intelligently fills in empty fields without overwriting existing data.
+- **Data Accumulation:** Accumulates (adds up) values for `total_spend` and loyalty `points` if the customer or loyalty card already exists in Odoo, rather than overwriting them.
- **High-Performance DataTables:** Uses Server-Side Processing for pagination, sorting, and debounced searching. This ensures the UI remains lightning-fast, even with hundreds of thousands of records.
- **Beautiful & Modern UI:** Features a clean, responsive "Glassmorphism" light theme optimized for both Desktop and Mobile environments.
-- **Secure Sessions:** Employee logins are secured with an 8-hour session lifetime mechanism.
+- **Secure MySQL Authentication:** Employee logins are handled via a separate MySQL database. Passwords are verified using encoding, with a fallback for a default unencrypted default password.
- **Interactive Prompts:** Integrated with SweetAlert2 to prevent accidental migrations and to provide clean success/error feedback.
## 🛠️ Tech Stack
- **Backend:** Python 3, Flask, Flask-Session
-- **Database Connection:** `psycopg2-binary` (PostgreSQL)
+- **Databases:**
+ - `psycopg2-binary` (PostgreSQL - for Member Data)
+ - `PyMySQL` (MySQL - for Employee Login)
- **Odoo Integration:** `xmlrpc.client` (Odoo XML-RPC API)
- **Frontend:** HTML5, CSS3, Vanilla JavaScript, jQuery, DataTables, SweetAlert2
@@ -39,10 +42,10 @@ A lightweight, modern web application built with Python (Flask) that streamlines
```
4. **Configure Credentials**
- Open `app.py` in your text editor and locate the **Configurations** section at the top of the file. Update the placeholders with your actual PostgreSQL and Odoo 19 credentials:
+ Open `app.py` in your text editor and locate the **Configurations** section at the top of the file. Update the placeholders with your actual PostgreSQL, MySQL, and Odoo 19 credentials:
```python
- # PostgreSQL
+ # PostgreSQL (Member Data)
DB_HOST = "localhost"
DB_PORT = "5432"
DB_NAME = "your_postgres_db"
@@ -50,10 +53,17 @@ A lightweight, modern web application built with Python (Flask) that streamlines
DB_PASS = "your_postgres_password"
# Odoo 19
- ODOO_URL = "localhost"
- ODOO_DB = "odoo19"
+ ODOO_URL = "http://localhost:8069"
+ ODOO_DB = "odoo19_db"
ODOO_USER = "admin"
ODOO_PASS = "admin"
+
+ # MySQL (Login DB)
+ MYSQL_HOST = "localhost"
+ MYSQL_PORT = 3306
+ MYSQL_DB = "klaim_db"
+ MYSQL_USER = "your_mysql_user"
+ MYSQL_PASS = "your_mysql_password"
```
5. **Run the Application**
@@ -70,21 +80,22 @@ When a migration is triggered, the following mapping occurs:
| PostgreSQL (`tada_member`) | Odoo 19 (`res.partner`) | Notes |
| :--- | :--- | :--- |
| `name` | `name` | |
-| `phone_number` | `phone` | |
-| `email` | `email` | |
+| `phone_number` | `phone` | Normalized for searching to prevent duplicates |
+| `email` | `email` | Updates only if Odoo field is empty |
| `gender` | `gender` | Capitalized automatically (e.g., `female` -> `Female`) |
-| `birthday` | `birth_date` | Converted to string format |
-| `city` | `city` | |
-| `total_spending` | `total_spend` | Converted to Float |
+| `birthday` | `birth_date` | Updates only if Odoo field is empty |
+| `city` | `city` | Updates only if Odoo field is empty |
+| `total_spending` | `total_spend` | Converted to Float. **Accumulates** if data exists. |
| `level` | `membership_level_id` | Linked via `loyalty.program` search (Many2one) |
### 2. `loyalty.card` (Membership Card)
| PostgreSQL (`tada_member`) | Odoo 19 (`loyalty.card`) | Notes |
| :--- | :--- | :--- |
-| `point_amount` | `points` | Converted to Float |
-| (Generated) | `partner_id` | Linked to the newly created `res.partner` ID |
+| `point_amount` | `points` | Converted to Float. **Accumulates** if card exists. |
+| (Generated) | `partner_id` | Linked to the matched/created `res.partner` ID |
| (Generated) | `program_id` | Linked to the matched `loyalty.program` ID |
## 🛡️ Important Notes
- **Data Validation:** The script automatically handles `NULL`/`None` values from PostgreSQL, safely converting them to `False` to prevent XML-RPC marshal errors when writing to Odoo.
- **Debounce Optimization:** The search bar waits 400ms after you stop typing and requires a minimum of 3 characters before querying the database, drastically reducing server load.
+- **Session Expiry:** Security sessions automatically expire after 8 hours of inactivity.
diff --git a/app.py b/app.py
index 6ac7494..af5d915 100644
--- a/app.py
+++ b/app.py
@@ -1,5 +1,8 @@
import xmlrpc.client
import psycopg2
+import pymysql
+import base64
+import re
from psycopg2.extras import RealDictCursor
from flask import Flask, render_template, request, session, redirect, url_for, jsonify
from datetime import datetime, timedelta
@@ -23,6 +26,34 @@ ODOO_DB = "odoo19_OT_v3"
ODOO_USER = "admin"
ODOO_PASS = "admin"
+# MySQL (Login DB)
+MYSQL_HOST = "192.169.0.13"
+MYSQL_PORT = 3306
+MYSQL_DB = "klaim_db"
+MYSQL_USER = "u8975009_root"
+MYSQL_PASS = "Jempol&1992Mutu_hosting"
+
+def get_mysql_connection():
+ return pymysql.connect(
+ host=MYSQL_HOST,
+ port=MYSQL_PORT,
+ user=MYSQL_USER,
+ password=MYSQL_PASS,
+ database=MYSQL_DB,
+ cursorclass=pymysql.cursors.DictCursor
+ )
+
+def normalize_phone(phone):
+ if not phone:
+ return ""
+ # Remove all non-digits
+ num = re.sub(r'\D', '', str(phone))
+ if num.startswith('62'):
+ num = num[2:]
+ elif num.startswith('0'):
+ num = num[1:]
+ return num
+
def get_db_connection():
conn = psycopg2.connect(
host=DB_HOST,
@@ -37,21 +68,55 @@ def get_db_connection():
def index():
if 'employee_id' not in session:
return redirect(url_for('login'))
- return render_template('index.html', employee_id=session['employee_id'])
+ return render_template('index.html', employee_name=session.get('employee_name', session['employee_id']))
@app.route('/login', methods=['GET', 'POST'])
def login():
+ error = None
if request.method == 'POST':
employee_id = request.form.get('employee_id')
- if employee_id:
- session.permanent = True
- session['employee_id'] = employee_id
- return redirect(url_for('index'))
- return render_template('login.html')
+ password = request.form.get('password')
+
+ if employee_id and password:
+ try:
+ conn = get_mysql_connection()
+ with conn.cursor() as cursor:
+ cursor.execute("SELECT EMP_FULL_NAME, EMP_PASS FROM m_employee WHERE EMP_F_NUM = %s", (employee_id,))
+ user = cursor.fetchone()
+
+ if user:
+ db_pass_encoded = user['EMP_PASS']
+
+ # Check default unencrypted password first
+ if password == "11111" and db_pass_encoded == "11111":
+ is_valid = True
+ else:
+ input_pass_encoded = base64.b64encode(password.encode('utf-8')).decode('utf-8')
+ is_valid = (input_pass_encoded == db_pass_encoded)
+
+ if is_valid:
+ session.permanent = True
+ session['employee_id'] = employee_id
+ session['employee_name'] = user['EMP_FULL_NAME']
+ return redirect(url_for('index'))
+ else:
+ error = "Invalid password."
+ else:
+ error = "Employee ID not found."
+ except Exception as e:
+ error = f"Database error: {str(e)}"
+ finally:
+ if 'conn' in locals() and conn.open:
+ conn.close()
+ else:
+ error = "Please provide both Employee ID and Password."
+
+ return render_template('login.html', error=error)
@app.route('/logout')
def logout():
session.pop('employee_id', None)
+ session.pop('employee_name', None)
return redirect(url_for('login'))
@app.route('/api/customers')
@@ -68,7 +133,7 @@ def get_customers():
order_column_index = request.args.get('order[0][column]', type=int, default=0)
order_dir = request.args.get('order[0][dir]', default='asc')
- columns_map = ['id', 'name', 'phone_number', 'email', 'level', 'point_amount', 'responsible', 'total_spending']
+ columns_map = ['id', 'name', 'phone_number', 'email', 'level', 'point_amount', 'responsible']
order_column = columns_map[order_column_index] if order_column_index < len(columns_map) else 'id'
if order_dir not in ['asc', 'desc']:
order_dir = 'asc'
@@ -151,7 +216,23 @@ def migrate_customer(customer_id):
models = xmlrpc.client.ServerProxy(f'{ODOO_URL}/xmlrpc/2/object')
- # 1. Find loyalty.program based on level
+ # 1. Normalize phone and check if partner exists
+ phone_raw = customer.get('phone_number') or ''
+ norm_phone = normalize_phone(phone_raw)
+
+ partner_id = False
+ existing_partner = None
+
+ if norm_phone:
+ partner_ids = models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'res.partner', 'search_read',
+ [[('phone', 'ilike', norm_phone)]],
+ {'fields': ['name', 'phone', 'email', 'gender', 'birth_date', 'city', 'total_spend', 'membership_level_id'], 'limit': 1}
+ )
+ if partner_ids:
+ existing_partner = partner_ids[0]
+ partner_id = existing_partner['id']
+
+ # 2. Find loyalty.program based on level
level = customer.get('level', '')
program_ids = models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'loyalty.program', 'search', [[('name', 'ilike', level)]])
@@ -165,32 +246,67 @@ def migrate_customer(customer_id):
if isinstance(gender_val, str) and gender_val:
gender_val = gender_val.title()
- # 2. Create res.partner
- partner_data = {
- 'name': customer.get('name', ''),
- 'phone': customer.get('phone_number', ''),
- 'email': customer.get('email', ''),
- 'gender': gender_val or '',
- 'birth_date': str(customer.get('birthday', '')) if customer.get('birthday') else False,
- 'city': customer.get('city', ''),
- 'membership_level_id': program_id,
- 'total_spend': float(customer.get('total_spending') or 0.0)
- }
+ pg_total_spend = float(customer.get('total_spending') or 0.0)
- # Replace None with False for Odoo XML-RPC compatibility
- partner_data = {k: (v if v is not None else False) for k, v in partner_data.items()}
+ if existing_partner:
+ # Update existing partner (fill empty fields)
+ update_data = {}
+ if not existing_partner.get('email') and customer.get('email'):
+ update_data['email'] = customer.get('email')
+ if not existing_partner.get('gender') and gender_val:
+ update_data['gender'] = gender_val
+ if not existing_partner.get('birth_date') and customer.get('birthday'):
+ update_data['birth_date'] = str(customer.get('birthday'))
+ if not existing_partner.get('city') and customer.get('city'):
+ update_data['city'] = customer.get('city')
+ if not existing_partner.get('membership_level_id'):
+ update_data['membership_level_id'] = program_id
+
+ # Accumulate total_spend
+ current_spend = float(existing_partner.get('total_spend') or 0.0)
+ if pg_total_spend > 0:
+ update_data['total_spend'] = current_spend + pg_total_spend
+
+ if update_data:
+ models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'res.partner', 'write', [[partner_id], update_data])
+ else:
+ # Create new partner
+ partner_data = {
+ 'name': customer.get('name', ''),
+ 'phone': phone_raw,
+ 'email': customer.get('email', ''),
+ 'gender': gender_val or '',
+ 'birth_date': str(customer.get('birthday', '')) if customer.get('birthday') else False,
+ 'city': customer.get('city', ''),
+ 'membership_level_id': program_id,
+ 'total_spend': pg_total_spend
+ }
+ # Replace None with False for Odoo XML-RPC compatibility
+ partner_data = {k: (v if v is not None else False) for k, v in partner_data.items()}
+ partner_id = models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'res.partner', 'create', [partner_data])
+
+ # 3. Handle loyalty.card
+ point_amount = float(customer.get('point_amount') or 0.0)
- partner_id = models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'res.partner', 'create', [partner_data])
+ card_ids = models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'loyalty.card', 'search_read',
+ [[('partner_id', '=', partner_id), ('program_id', '=', program_id)]],
+ {'fields': ['points'], 'limit': 1}
+ )
- # 3. Create loyalty.card
- point_amount = customer.get('point_amount')
- card_data = {
- 'partner_id': partner_id,
- 'program_id': program_id,
- 'points': float(point_amount) if point_amount is not None else 0.0
- }
-
- card_id = models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'loyalty.card', 'create', [card_data])
+ if card_ids:
+ # Accumulate points
+ existing_card = card_ids[0]
+ current_points = float(existing_card.get('points') or 0.0)
+ if point_amount > 0:
+ models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'loyalty.card', 'write', [[existing_card['id']], {'points': current_points + point_amount}])
+ else:
+ # Create new card
+ card_data = {
+ 'partner_id': partner_id,
+ 'program_id': program_id,
+ 'points': point_amount
+ }
+ models.execute_kw(ODOO_DB, uid, ODOO_PASS, 'loyalty.card', 'create', [card_data])
# 4. Update tada_member
now = datetime.now()
diff --git a/requirements.txt b/requirements.txt
index 3be58eb..0e10bbf 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -2,6 +2,4 @@ Flask==3.0.0
Flask-Session==0.5.0
psycopg2-binary==2.9.9
Werkzeug==3.0.1
-pandas
-sqlalchemy
-openpyxl
\ No newline at end of file
+PyMySQL==1.1.0
diff --git a/templates/index.html b/templates/index.html
index 49e8db3..0a28dd2 100644
--- a/templates/index.html
+++ b/templates/index.html
@@ -9,7 +9,7 @@
Tada Migration
- Enter your catapa ID to securely access the migration portal.
+ Enter your Catapa ID to securely access the migration portal.
+
+ {% if error %}
+
+ {{ error }}
+
+ {% endif %}