aboutsummaryrefslogtreecommitdiffstats
path: root/packages/pipeline/src
Commit message (Collapse)AuthorAgeFilesLines
* Fix bug that occurs when token metadata decimals is undefined. (#1426)Alex Browne2018-12-132-4/+18
| | | | | | * Fix bug that occurs when token metadata decimals is undefined. * move toBigNumberOrNull to utils
* Fix linter errorsLeonid Logvinov2018-12-138-22/+22
|
* Make pull_missing_blocks script consider all events with block numbers (#1420)Alex Browne2018-12-121-1/+14
|
* track idex orderbook snapshots (#1397)zkao2018-12-129-20/+490
| | | * Track Idex and Oasis Orderbook Snapshots
* Refactor event scraping and add support for scraping ERC20 approval events ↵Alex Browne2018-12-1212-194/+421
| | | | | | | | (#1401) * Refactor event scraping and add support for scraping ERC20 approval events * Add tests for data_sources/contract-wrappers/utils
* perform null check on token metadata (#1412)Xianny2018-12-111-1/+1
|
* Fix/pipeline/ohlcv ratelimit (#1403)Xianny2018-12-112-20/+18
|\ | | | | | | Use time-based throttling and increase batch size of CC query
| * change to camelCasexianny2018-12-112-3/+3
| |
| * lint: remove unused variablexianny2018-12-081-1/+0
| |
| * refresh rate limit every second instead of every minutexianny2018-12-081-3/+3
| |
| * cleanup: stray import, rename variablexianny2018-12-081-4/+3
| |
| * query CC with larger batch sizexianny2018-12-081-3/+4
| |
| * rename variable and define default in only 1 locationxianny2018-12-082-4/+5
| |
| * upgrade throttling codexianny2018-12-081-13/+11
| |
* | Implement prefer-template tslint rulekao2018-12-082-4/+4
|/
* Fix/pipeline/ohlcv (#1393)Xianny2018-12-062-31/+43
| | | The OHLCV script in data pipeline quits early when we get no data from Crypto Compare. Sometimes Crypto Compare gives us a valid empty response (e.g. when we query for way back in time) and we need to just continue. This adds better filtering for the types of Crypto Compare responses to detect when we should continue and when we should really quit.
* Fix RadarRelay timestamps (#1391)Alex Browne2018-12-062-4/+8
| | | | | | * Fixing rr timestamps * Apply prettier
* Change type of transactions.gas_used and gas_price to BigNumber/numericAlex Browne2018-12-052-7/+9
|
* Fix prettierAlex Browne2018-12-051-18/+11
|
* Address PR feedbackAlex Browne2018-12-0517-62/+63
|
* Use a string template in packages/pipeline/src/scripts/pull_missing_events.tsFabio B2018-12-051-1/+1
| | | Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
* Fix typo in packages/pipeline/src/parsers/bloxy/index.tsFabio B2018-12-051-1/+1
| | | Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
* Fix linterAlex Browne2018-12-051-1/+1
|
* pull OHLCV records from Crypto Compare (#1349)Xianny2018-12-058-16/+344
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * [WIP] pull OHLCV records from Crypto Compare * lint * refactor to pull logic out of script and into modules * add entity test for ohlcv_external entity * implement rate limit and chronological backfill for ohlcv * add unit tests; cleanup variable names * Fetch OHLCV pairs params from events table * better method names * fix outdated test * lint * Clean up after review * oops * fix failing test * better filtering of most recent records * fix bug when generating pairs * fix default earliest backfill date * fix bug with retrieving backfill time * prettier
* Token_orderbook_snapshots for Ddex and Paradex(#1354)zkao2018-12-0510-0/+488
| | | | | | * Implements the TokenOrderbookSnapshot Table * Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
* Add script for parsing competing dex trades from Bloxy (#1355)Alex Browne2018-12-056-4/+298
|
* Introduce framework for running basic tests for entities (#1344)Alex Browne2018-12-0511-29/+70
| | | | | | | | | | | | | | | | | | | | | | * Introduce framework for running basic tests for entities * Add pipeline tests to CircleCI config * Make pipeline tests more configurable and fix CircleCI config * Add coverage dir to pipeline package * Add basic tests for all exchange event entities * Add tests for remaining entities * Create separate test scripts in package.json and add new info to README * Update db_setup.ts to revert migrations even if you are using docker * Automatically pull the postgres image if needed * Add comment about why NumberToBigIntTransformer is needed
* Reduce number of blocks per query in pull_missing_eventsAlex Browne2018-12-051-1/+1
|
* updating comment for 0x trusted tokensJake Ellowitz2018-12-053-8/+7
|
* metadata and trusted sources in same raw tableJake Ellowitz2018-12-059-53/+55
|
* Pull token metadata re trusted tokensJake Ellowitz2018-12-0511-2/+162
|
* Add script for pulling missing block dataAlex Browne2018-12-053-3/+89
|
* Check for special characters in table name in pull_missing_eventsAlex Browne2018-12-051-3/+7
|
* Add support for pulling Cancel and CancelUpTo eventsAlex Browne2018-12-054-87/+150
|
* Add transactionHash to CancelUpToEvent and CancelEvent entitiesAlex Browne2018-12-053-1/+6
|
* Optimize database operations in pull_missing_events scriptAlex Browne2018-12-051-9/+26
|
* Add homepage_url to relayersAlex Browne2018-12-052-7/+8
|
* Add workaround for broken save methodAlex Browne2018-12-052-2/+22
|
* Fix chunk size in pull_missing_eventsAlex Browne2018-12-051-1/+1
|
* Use built-in chunk feature of TypeORM save methodAlex Browne2018-12-051-7/+2
|
* Change some column types from varchar to numericAlex Browne2018-12-058-35/+57
|
* Add raw schema prefix to query in pull_missing_eventsAlex Browne2018-12-051-1/+1
|
* Rename taker and maker amounts in sra_ordersAlex Browne2018-12-051-2/+2
|
* Implement scraping sra orders from radar relayAlex Browne2018-12-056-22/+52
|
* Preliminary work for adding RR order book scrapingAlex Browne2018-12-051-0/+52
|
* Update schema for sra_ordersAlex Browne2018-12-051-6/+1
|
* Pointing entities to raw schemaJake Ellowitz2018-12-058-9/+9
| | | | Fix linter issues
* Mapping dev schema to initial migrationJake Ellowitz2018-12-053-1/+18
|
* Configure linter with --format stylish and fix linter errorsAlex Browne2018-12-0511-6/+80
|
* Configure TypeORM for migrations. Add new package.json scripts.Alex Browne2018-12-053-26/+11
|
* Rename Transaction.ts to transaction.tsAlex Browne2018-12-053-0/+0
| | | | | | Rename Relayer.ts to relayer.ts Rename Block.ts to block.ts
* Add better error handling for immediately invoked async functionsAlex Browne2018-12-054-85/+20
|
* Rename table and column namesAlex Browne2018-12-059-97/+171
|
* Reorganize entities. Make scripts work from any directory.Alex Browne2018-12-0513-12/+36
|
* Split index.ts into multiple scripts in scripts/ and detect last known block ↵Alex Browne2018-12-054-92/+95
| | | | when pulling events
* Remove unused function parameter in index.ts scriptAlex Browne2018-12-051-2/+2
|
* Use bigint for gasUsed and gasPrice in TransactionAlex Browne2018-12-052-5/+7
|
* Update relayer code to use new relayer-registry formatAlex Browne2018-12-054-6/+14
|
* Implement fetching and parsing relayer infoAlex Browne2018-12-054-6/+106
|
* Implement support for getting and parsing blocks and transactionsAlex Browne2018-12-055-0/+149
|
* chore: Fix tslint config.Alex Browne2018-12-051-1/+1
|
* Fix bugs in merge_v2_events script and disable verbose loggingAlex Browne2018-12-052-2/+5
|
* Enable verbose loggingAlex Browne2018-12-051-1/+1
|
* Rename index.ts -> merge_v2_eventsAlex Browne2018-12-051-0/+0
|
* Update script to work with existing v1 pipeline dataAlex Browne2018-12-058-36/+84
|
* Update pipeline to use new @0x npm org for importsAlex Browne2018-12-055-12/+12
|
* Update to use ContractWrappers + Infura instead of EtherscanAlex Browne2018-12-056-135/+71
|
* Rebase and update to use new modularized artifactsAlex Browne2018-12-054-2279/+2
|
* Add tests for SRA order conversionAlex Browne2018-12-052-4/+4
|
* Add soruceUrl to SraOrder entityAlex Browne2018-12-052-1/+4
|
* Add preliminary support for scraping orders from SRA endpoints (no ↵Alex Browne2018-12-057-12/+133
| | | | pagination, only RR support for now)
* For event entities, rename address -> contractAddress and make it a primary keyAlex Browne2018-12-055-11/+19
|
* Implement scraping and parsing exchange CancelUpTo eventsAlex Browne2018-12-053-9/+45
|
* Use multi-primary keys for event and filter null logIndexesAlex Browne2018-12-054-6/+15
|
* Implement scraping and parsing exchange cancel eventsAlex Browne2018-12-055-10/+81
|
* Make event parsing more functional and less class-basedAlex Browne2018-12-053-65/+21
|
* Re-organize event parsing and decodingAlex Browne2018-12-056-89/+120
|
* Add support for decoding asset dataAlex Browne2018-12-054-36/+92
|
* Add typeorm and begin inserting database recordsAlex Browne2018-12-054-7/+78
|
* Make contractAbi a parameter of getContractEventsAsyncAlex Browne2018-12-053-10/+26
|
* Add tests for etherscan eventsAlex Browne2018-12-052-9/+17
|
* Restructure pipeline package. Create data-sources dirAlex Browne2018-12-054-75/+96
|
* Remove old code. Create function for getting contract events via etherscanAlex Browne2018-12-0522-1868/+2348
|
* Rebase pipeline branch off developmentAlex Browne2018-12-0519-0/+1870
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-03 10:18:27 +0800