diff options
author | Fabio Berger <me@fabioberger.com> | 2019-01-12 00:12:55 +0800 |
---|---|---|
committer | Fabio Berger <me@fabioberger.com> | 2019-01-12 00:12:55 +0800 |
commit | 96ff5eef644b8c4d3f8fc4ec53270991fb51f121 (patch) | |
tree | 530f43b03d2b3aa09b5f6552d4285df92d45fed0 /packages/pipeline/README.md | |
parent | 7f5a2c972bb86df77c423fb7029e9a629ecc1ede (diff) | |
parent | 2cf57a48dd2857dd5cf2f31f4c60dd47ae4d34a5 (diff) | |
download | dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.tar dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.tar.gz dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.tar.bz2 dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.tar.lz dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.tar.xz dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.tar.zst dexon-sol-tools-96ff5eef644b8c4d3f8fc4ec53270991fb51f121.zip |
merge development
Diffstat (limited to 'packages/pipeline/README.md')
-rw-r--r-- | packages/pipeline/README.md | 32 |
1 files changed, 25 insertions, 7 deletions
diff --git a/packages/pipeline/README.md b/packages/pipeline/README.md index 794488cac..4fc8e0ff9 100644 --- a/packages/pipeline/README.md +++ b/packages/pipeline/README.md @@ -147,20 +147,38 @@ set the`ZEROEX_DATA_PIPELINE_DB_URL` environment variable to a valid starting from that block number. 7. Run the migrations and then run your new script locally and verify it works as expected. +8. After all tests pass and you can run the script locally, open a new PR to + the monorepo. Don't merge this yet! +9. If you added any new scripts or dependencies between scripts, you will need + to make changes to https://github.com/0xProject/0x-pipeline-orchestration + and make a separate PR there. Don't merge this yet! +10. After your PR passes code review, ask @feuGeneA or @xianny to deploy your + changes to the QA environment. Check the [QA Airflow dashboard](http://airflow-qa.0x.org:8080) + to make sure everything works correctly in the QA environment. +11. Merge your PR to 0x-monorepo (and + https://github.com/0xProject/0x-pipeline-orchestration if needed). Then ask + @feuGeneA or @xianny to deploy to production. +12. Monitor the [production Airflow dashboard](http://airflow.0x.org:8080) to + make sure everything still works. +13. Celebrate! :tada: #### Additional guidelines and tips: -* Table names should be plural and separated by underscores (e.g., +- Table names should be plural and separated by underscores (e.g., `exchange_fill_events`). -* Any table which contains data which comes directly from a third-party source +- Any table which contains data which comes directly from a third-party source should be namespaced in the `raw` PostgreSQL schema. -* Column names in the database should be separated by underscores (e.g., +- Column names in the database should be separated by underscores (e.g., `maker_asset_type`). -* Field names in entity classes (like any other fields in TypeScript) should +- Field names in entity classes (like any other fields in TypeScript) should be camel-cased (e.g., `makerAssetType`). -* All timestamps should be stored as milliseconds since the Unix Epoch. -* Use the `BigNumber` type for TypeScript code which deals with 256-bit +- All timestamps should be stored as milliseconds since the Unix Epoch. +- Use the `BigNumber` type for TypeScript code which deals with 256-bit numbers from smart contracts or for any case where we are dealing with large floating point numbers. -* [TypeORM documentation](http://typeorm.io/#/) is pretty robust and can be a +- [TypeORM documentation](http://typeorm.io/#/) is pretty robust and can be a helpful resource. + +* Scripts/parsers should perform minimum data transformation/normalization. + The idea here is to have a raw data feed that will be cleaned up and + synthesized in a separate step. |