aboutsummaryrefslogtreecommitdiffstats
path: root/packages/pipeline/src/data_sources
Commit message (Collapse)AuthorAgeFilesLines
* Refactor event scraping and add support for scraping ERC20 approval events ↵Alex Browne2018-12-123-51/+137
| | | | | | | | (#1401) * Refactor event scraping and add support for scraping ERC20 approval events * Add tests for data_sources/contract-wrappers/utils
* Fix/pipeline/ohlcv ratelimit (#1403)Xianny2018-12-111-17/+15
|\ | | | | | | Use time-based throttling and increase batch size of CC query
| * change to camelCasexianny2018-12-111-2/+2
| |
| * lint: remove unused variablexianny2018-12-081-1/+0
| |
| * refresh rate limit every second instead of every minutexianny2018-12-081-3/+3
| |
| * cleanup: stray import, rename variablexianny2018-12-081-4/+3
| |
| * query CC with larger batch sizexianny2018-12-081-3/+4
| |
| * rename variable and define default in only 1 locationxianny2018-12-081-2/+3
| |
| * upgrade throttling codexianny2018-12-081-13/+11
| |
* | Implement prefer-template tslint rulekao2018-12-082-4/+4
|/
* Fix/pipeline/ohlcv (#1393)Xianny2018-12-061-9/+27
| | | The OHLCV script in data pipeline quits early when we get no data from Crypto Compare. Sometimes Crypto Compare gives us a valid empty response (e.g. when we query for way back in time) and we need to just continue. This adds better filtering for the types of Crypto Compare responses to detect when we should continue and when we should really quit.
* Address PR feedbackAlex Browne2018-12-051-1/+2
|
* pull OHLCV records from Crypto Compare (#1349)Xianny2018-12-051-0/+94
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * [WIP] pull OHLCV records from Crypto Compare * lint * refactor to pull logic out of script and into modules * add entity test for ohlcv_external entity * implement rate limit and chronological backfill for ohlcv * add unit tests; cleanup variable names * Fetch OHLCV pairs params from events table * better method names * fix outdated test * lint * Clean up after review * oops * fix failing test * better filtering of most recent records * fix bug when generating pairs * fix default earliest backfill date * fix bug with retrieving backfill time * prettier
* Token_orderbook_snapshots for Ddex and Paradex(#1354)zkao2018-12-052-0/+170
| | | | | | * Implements the TokenOrderbookSnapshot Table * Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
* Add script for parsing competing dex trades from Bloxy (#1355)Alex Browne2018-12-051-0/+133
|
* Reduce number of blocks per query in pull_missing_eventsAlex Browne2018-12-051-1/+1
|
* Pull token metadata re trusted tokensJake Ellowitz2018-12-051-0/+29
|
* Add support for pulling Cancel and CancelUpTo eventsAlex Browne2018-12-051-10/+40
|
* Configure linter with --format stylish and fix linter errorsAlex Browne2018-12-053-5/+5
|
* Update relayer code to use new relayer-registry formatAlex Browne2018-12-051-2/+2
|
* Implement fetching and parsing relayer infoAlex Browne2018-12-051-0/+33
|
* Implement support for getting and parsing blocks and transactionsAlex Browne2018-12-051-0/+22
|
* Update script to work with existing v1 pipeline dataAlex Browne2018-12-051-1/+4
|
* Update pipeline to use new @0x npm org for importsAlex Browne2018-12-051-3/+3
|
* Update to use ContractWrappers + Infura instead of EtherscanAlex Browne2018-12-052-52/+51
|
* Re-organize event parsing and decodingAlex Browne2018-12-051-0/+52