Spot Connector QA Checklist

Before approving new connectors, the Hummingbot Foundation Qualtiy Assurance (QA) team will do test pull requests to ensure it is working as expected. Below is our test template for spot connectors.

Title	Steps	Expected result
Connecting an API key	Run connect name-of-connector Sample: connect binance	1. Connects when a valid API key is added. 2. Throws an error or warning if the API key is invalid, expired, or has other issues. 3. The same API key can be used on multiple bots unless specified that can only be used in one instance at a time.
Pulling the balance	Run balance command	1. Displays the current available balance and should match the balance shown in the exchange. 2. The allocation should be updated whenever there is an open order.
Market availability during the strategy creation	While setting up strategy, you will be asked: Enter the token trading pair you would like to trade on name-of-connector (e.g. ZRX-ETH) >>>	1. Autocomplete lists should be available when setting a market during the strategy creation. 2. All markets should work when the created strategy is started.
Compatibility with the available strategies	Create a simple config on different hummingbot strategies	1. The connector should work on any of the strategies available in the client unless the connector is intended for a specific strategy. 2. The connector should work both as a maker and taker exchange (spot connectors only)
Price and balance updates	Run the status or status --live command Shortcut key for status command ctrl+s	1. In the status window, the prices are constantly being updated whenever a change takes place in the order book. 2. Available balances are updated whenever an order is created or canceled. 3. A consistent number of orders are created except if there are multiple bots using the asset, there are hanging orders or an order is removed due to a specific parameter.
Order submission and cancellation	Setup a simple market-making strategy and start the bot	1. Created orders must include an order ID with broker prefix if there is. 2. Orders are submitted without any error in the client. 3. Submitted order should match the information of the open order in the exchange. 4. Orders are canceled without any error. 5. Orders are not getting stuck or left out unless it is a manual order. 6. The client should not cancel orders that are not created within the instance such as manual orders, orders created by other instances, or third-party bots. 7. Gracefully rejects/cancels orders that don’t pass the exchange rule.
Fast refresh rate	1. Create a pure market-making strategy and set a fast order refresh time with order refresh tolerance disabled - to disable, config order_refresh_tolerance -1 2. Run status or status --live command 3. Crosscheck the client and exchange on open orders. - If there are filled order, check trade history on exchange and history or history --verbose command on client Note: Make sure to run the client on a machine with stable internet connection. Ideally, you would want to test this on a Linux cloud server	1. Gracefully cancels orders: No stuck or lost orders 2. Balance updates accordingly 3. There should be no error in the logs during the fast cancellation. 4. Filled orders are tracked and saved if a filled event took place. 5. Rate limit warnings are thrown whenever the request is close to maximizing the allowed limit. 6. Stops placing an order when the rate limit is reached but maintains the connectivity with the exchange.
Long refresh rate	1. Create a pure market-making strategy and set a fast order refresh time with order refresh tolerance disabled - to disable, config order_refresh_tolerance -1 2. Run status or status --live command Note: Make sure to run the client on a machine with stable internet connection. Ideally, you would want to test this on a Linux cloud server	1. Maintains connectivity when there is an open order. 2. Gracefully cancel an order when not filled. 3. No error should come up during the period that an order is opened (exceptions if there are network issues) 4. If a network issue took place, the open orders should be tracked once the connection is established. 5. Tracked and save filled events that may happen during a disconnection.
Multiple orders	Create a market-making strategy that support multiple order levels - set config order_level	1. Simultaneously submitting multiple orders without any error. 2. Simultaneously cancels all the orders without any error. 3. Available balance and allocations are adjusted accordingly. 4. Order amounts and levels are adjusted accordingly based on the available balance.
Hanging orders	Create a market-making strategy that support hanging orders and set cancellation percentage - set config haning_orders_enabled - set config hanging_orders_cancel_pct	1. Continuously tracked hanging orders created within the instance. 2. Tracked and saved a filled hanging order 3. Hanging order should be canceled whenever the strategy is stopped or reached the cancelation percentage. 4. Hanging orders remain uncanceled whenever non-hanging orders are refreshed. 5. No hanging order duplication
Multiple bots	1. Setup multiple test bots using the same exchange ( same account and API key) 2. Start all test bots. Best to setup with docker build 3. Set tight spreads enough to trigger filled trade event on one of the test bots 4. Review log pane 5. Compare data of all test bots by running history or history --verbose command, CSV or sqlite file and exchange’s trade history	1. Open orders in each bot should not be affected by cancelation events happening in another bot. 2. There should be no conflicting order IDs
Data integrity	1. Run status --live 2, Run order_book --live - you can add an argument of order_book --live --line 10 where it should display 10 orders per side of the orderbook 3. Run ticker --live	1. Orders book in the client is in sync with the order book in the exchange. 2. Constantly update whenever there is a change in the exchange. 3. Prices are updated constantly (status and ticker) whenever there is a change in the exchange.
Filled event	Setup a market-making bot with spreads tight enough for filled trade event. 1. Run history command while there are no field orders 2. Run history or history --verbose command after getting a fill - You can increase the decimal value by setting up history --verbose --precision 5 where 5 is the added number of decimal 3. Review history --verbose command, CSV or sqlite file and exchange’s trade history	1. Full and partial fills are both tracked and recorded properly. 2. Filled order information should match the trade history in the exchange.
History	Run history command with or without trades	1. Correctly display filled order information - No duplicate orders - Save partial fills - Display only orders created by the instance 2. Correctly displays asset information
Trade fees	After getting trades on the client: 1. Run history --verbose command 2. Record total fee 3. Login to the exchange website and check the trade history 4. Manually compute the fees of all the filled trades done by the bot	1. Should use the trade fee if available otherwise an estimate fee is used. 2. The transaction fee recorded in the client (CSV or SQLite) should match the fee shown in the trade history in the exchange.
Data aggregation [For QA foundation only]	1. Set a tight spread to easily get a fill and check datadog - hummingbot usdt volume metrics after 15mins interval - you can set the anonymized_metrics_interval_min to a lower timeframe on conf/config_client.yml 2. Set a very tight spread to immediately get a fill then stop the bot without waiting for the heartbeat interval to complete.	1. When enabled, all filled trades are aggregated in Datadog