! PRE-RELEASE DEMO DOCUMENTATION !

GitLab Configurator Updates:

File: global/config.yaml
Paste the code below into the position indicated on the screen:

# at the top under DB
clickhouse:
  image: clickhouse/clickhouse-server:25.1.2

# ...


operations: gitlab.tunex.io:5050/marionette-develop/core/operations:latest
balances: gitlab.tunex.io:5050/marionette-develop/core/balances:latest
accounting: gitlab.tunex.io:5050/marionette-develop/core/accounting:latest
trading:
  enabled: true
  image: gitlab.tunex.io:5050/marionette-develop/core/trading:latest
  separate_trading_user_account: false
  default_market: "ETH-USDT"

Docker Compose Templates:

File: templates/compose/required.yaml.t
Paste the code below into the position indicated on the screen:

    clickhouse:
            image: {{clickhouse.image}}
            volumes:
                - ../data/clickhouse_data/:/var/lib/clickhouse/
                - ../data/clickhouse_logs/:/var/log/clickhouse-server/
            ulimits:
                nproc: 65535
                nofile:
                    soft: 262144
                    hard: 262144

File: templates/compose/backend.yaml.t
Paste the code below into the position indicated on the screen:

Replace the whole operations.

    operations:
        restart: always
        image: {{components.operations}}
        env_file:
            - ../config/database.env
        environment:
            CANCEL_OVERAMOUNT: 1
{{#ifEquals components.trading.enabled true}}
            SEPARATE_TRADING_USER_ACCOUNT: {{#ifEquals components.trading.separate_trading_user_account true}}1{{else}}0{{/ifEquals}} 
{{/ifEquals}}
        depends_on:
            - clickhouse
{{#ifEquals mode "swarm"}}
        deploy:
            labels:
                - "traefik.http.routers.operations.rule=Host(`{{base_url}}`) && PathPrefix(`/components/operations`)"
                - "traefik.enable=true"
                - "traefik.http.services.operations.loadbalancer.server.port=3000"
                {{#if components.traefik.ssl}}
                - "traefik.http.routers.operations.entrypoints=websecure"
                - "traefik.http.routers.operations.tls=true"
                - "traefik.http.routers.operations.tls.certresolver=myresolver"
                - "traefik.http.routers.operations.middlewares=secureheaders"
                {{else}}
                - "traefik.http.routers.operations.entrypoints=web"
                {{/if}}
{{/ifEquals}}
{{#ifEquals mode "compose"}}
        labels:
            - "traefik.http.routers.operations.rule=Host(`{{base_url}}`) && PathPrefix(`/components/operations`)"
            - "traefik.enable=true"
            - "traefik.http.services.operations.loadbalancer.server.port=3000"
            {{#if components.traefik.ssl}}
            - "traefik.http.routers.operations.entrypoints=websecure"
            - "traefik.http.routers.operations.tls=true"
            - "traefik.http.routers.operations.tls.certresolver=myresolver"
            - "traefik.http.routers.operations.middlewares=secureheaders"
            {{else}}
            - "traefik.http.routers.operations.entrypoints=web"
            {{/if}}
{{/ifEquals}}

File: templates/compose/backend.yaml.t
Paste the code below into the position indicated on the screen:

Replace the balances

  balances:
        restart: always
        image: {{components.balances}}
        env_file:
            - ../config/database.env
        environment:
            LOGLEVEL: {{log_level}}
            NODE_OPTIONS: --max-old-space-size=14000
        depends_on:
            - clickhouse
{{#ifEquals mode "swarm"}}
        deploy:
            labels:
                - "traefik.http.routers.balances.rule=Host(`{{base_url}}`) && PathPrefix(`/components/balances`)"
                - "traefik.enable=true"
                - "traefik.http.services.balances.loadbalancer.server.port=3000"
                {{#if components.traefik.ssl}}
                - "traefik.http.routers.balances.entrypoints=websecure"
                - "traefik.http.routers.balances.tls=true"
                - "traefik.http.routers.balances.tls.certresolver=myresolver"
                - "traefik.http.routers.balances.middlewares=secureheaders"
                {{else}}
                - "traefik.http.routers.balances.entrypoints=web"
                {{/if}}
{{/ifEquals}}
{{#ifEquals mode "compose"}}
        labels:
            - "traefik.http.routers.balances.rule=Host(`{{base_url}}`) && PathPrefix(`/components/balances`)"
            - "traefik.enable=true"
            - "traefik.http.services.balances.loadbalancer.server.port=3000"
            {{#if components.traefik.ssl}}
            - "traefik.http.routers.balances.entrypoints=websecure"
            - "traefik.http.routers.balances.tls=true"
            - "traefik.http.routers.balances.tls.certresolver=myresolver"
            - "traefik.http.routers.balances.middlewares=secureheaders"
            {{else}}
            - "traefik.http.routers.balances.entrypoints=web"
            {{/if}}
{{/ifEquals}}

File: templates/compose/backend.yaml.t
Paste the code below into the position indicated on the screen:

Replace the trading and add the accounting in the code bellow.

    trading:
        restart: always
        image: {{components.trading.image}}
        env_file:
            - ../config/database.env
        environment:
            LOGLEVEL: {{log_level}}
            SEPARATE_TRADING_USER_ACCOUNT: {{#ifEquals components.trading.separate_trading_user_account true}}1{{else}}0{{/ifEquals}}
        depends_on:
            - clickhouse
{{#ifEquals mode "swarm"}}
        deploy:
            labels:
                - "traefik.http.routers.trading.rule=Host(`{{base_url}}`) && PathPrefix(`/components/trading`)"
                - "traefik.enable=true"
                - "traefik.http.services.trading.loadbalancer.server.port=3000"
                {{#if components.traefik.ssl}}
                - "traefik.http.routers.trading.entrypoints=websecure"
                - "traefik.http.routers.trading.tls=true"
                - "traefik.http.routers.trading.tls.certresolver=myresolver"
                - "traefik.http.routers.trading.middlewares=secureheaders"
                {{else}}
                - "traefik.http.routers.trading.entrypoints=web"
                {{/if}}
{{/ifEquals}}
{{#ifEquals mode "compose"}}
        labels:
            - "traefik.http.routers.trading.rule=Host(`{{base_url}}`) && PathPrefix(`/components/trading`)"
            - "traefik.enable=true"
            - "traefik.http.services.trading.loadbalancer.server.port=3000"
            {{#if components.traefik.ssl}}
            - "traefik.http.routers.trading.entrypoints=websecure"
            - "traefik.http.routers.trading.tls=true"
            - "traefik.http.routers.trading.tls.certresolver=myresolver"
            - "traefik.http.routers.trading.middlewares=secureheaders"
            {{else}}
            - "traefik.http.routers.trading.entrypoints=web"
            {{/if}}
{{/ifEquals}}

    accounting:
        restart: always
        image: {{components.accounting}}
        env_file:
            - ../config/database.env
        depends_on:
            - clickhouse
{{#ifEquals mode "swarm"}}
        deploy:
            labels:
                - "traefik.http.routers.accounting.rule=Host(`{{base_url}}`) && PathPrefix(`/components/accounting`)"
                - "traefik.enable=true"
                - "traefik.http.services.accounting.loadbalancer.server.port=3000"
                {{#if components.traefik.ssl}}
                - "traefik.http.routers.accounting.entrypoints=websecure"
                - "traefik.http.routers.accounting.tls=true"
                - "traefik.http.routers.accounting.tls.certresolver=myresolver"
                - "traefik.http.routers.accounting.middlewares=secureheaders"
                {{else}}
                - "traefik.http.routers.accounting.entrypoints=web"
                {{/if}}
{{/ifEquals}}
{{#ifEquals mode "compose"}}
        labels:
            - "traefik.http.routers.accounting.rule=Host(`{{base_url}}`) && PathPrefix(`/components/accounting`)"
            - "traefik.enable=true"
            - "traefik.http.services.accounting.loadbalancer.server.port=3000"
            {{#if components.traefik.ssl}}
            - "traefik.http.routers.accounting.entrypoints=websecure"
            - "traefik.http.routers.accounting.tls=true"
            - "traefik.http.routers.accounting.tls.certresolver=myresolver"
            - "traefik.http.routers.accounting.middlewares=secureheaders"
            {{else}}
            - "traefik.http.routers.accounting.entrypoints=web"
            {{/if}}
{{/ifEquals}}

Server Migration:

This release introduces a new database, ClickHouse. To ensure a successful migration, follow the steps below for the initial setup.

Migration Process:

Step 1: Start the migration. Allow the accounting container to migrate with the db-bridge. Wait approximately 30 minutes before proceeding.
Run the following commands:

Bring down the current setup:
docker-compose down
Start the required services:
docker-compose up -d nats db clickhouse

Start the db-bridge container and monitor the logs:

docker-compose up -d db-bridge && docker-compose logs -f db-bridge

Start the accounting container:
docker-compose up -d accounting

If the migration fails:
Run this command to remove all ClickHouse data and retry:

docker-compose rm -fs accounting clickhouse && rm -rf data/clickhouse_* && docker-compose up -d clickhouse && docker-compose up -d accounting

Step 2: Verify the migration for the balances container. Ensure there are no errors in the logs.
Run the following commands:

docker-compose up -d balances
docker-compose logs -f balances

Step 3: If everything is functioning correctly, start all containers:

docker-compose up -d

Re-migration.

If you saw the error this is how you do migration manually.

Chcek puppeteer and remember balance of someone.
start accounting
open clickhouse client, then check if the tables exist
delete from ledgerRecords where true
open logs for accounting in other terminal
docker compose logs -f accounting
In new windows run the marionette cli tool. ./m cli
run the migration
call 'accounting.migrate'
make sure it is finished, check the logs
if no errors migration is done
restart balances
docker compose restart balances
check puppeteer for balances and make sure the balances are same and there are not duplicate records