Generic Data Porting Server Implementation

Architecture & Structure

• Use a modular folder structure that separates concerns:
  • config: Configuration files (default, production).
  • src/controllers: Handle business logic and requests.
  • src/models: MongoDB schema definitions.
  • src/services: Specific tasks (APIService, CSVService, ExcelService, TransactionService, MongoDBService, Logger).
  • src/utils: Common utilities (dateUtils, validationUtils).
  • src/api/middleware: Express middleware.
  • test: Unit and integration tests.
  • scripts: Operational scripts (e.g., migration).
  • docs: Documentation.

Communication & Style Preferences

• Use JSDoc for detailed code documentation.
• Ensure code is scalable, robust, and generic enough to be reused across different projects.
• Maintain consistent coding style and indentation (e.g., using Biome/ESLint configurations).

Anti-Patterns

• Do not hardcode transaction names or API endpoints; use configuration files.
• Do not process documents that are already marked as processed or exist in the database without checking.

Triggers

• create a data porting server
• port excel csv to mongodb
• generic data migration tool
• process transactions and send to api
• nodejs data ingestion service