Commit Graph

9883 Commits

Author SHA1 Message Date
fragosti
2d6c46b83c feat: add copy code feature 2018-12-06 12:06:35 +01:00
Xianny
6f5787b2c4 pull OHLCV records from Crypto Compare (#1349)
* [WIP] pull OHLCV records from Crypto Compare

* lint

* refactor to pull logic out of script and into modules

* add entity test for ohlcv_external entity

* implement rate limit and chronological backfill for ohlcv

* add unit tests; cleanup variable names

* Fetch OHLCV pairs params from events table

* better method names

* fix outdated test

* lint

* Clean up after review

* oops

* fix failing test

* better filtering of most recent records

* fix bug when generating pairs

* fix default earliest backfill date

* fix bug with retrieving backfill time

* prettier
2018-12-06 12:06:34 +01:00
zkao
f96711bac3 Token_orderbook_snapshots for Ddex and Paradex(#1354)
* Implements the TokenOrderbookSnapshot Table

* Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
2018-12-06 12:05:38 +01:00
Alex Browne
ddd246a945 Add script for parsing competing dex trades from Bloxy (#1355) 2018-12-06 12:05:38 +01:00
Alex Browne
d14d38dabd Introduce framework for running basic tests for entities (#1344)
* Introduce framework for running basic tests for entities

* Add pipeline tests to CircleCI config

* Make pipeline tests more configurable and fix CircleCI config

* Add coverage dir to pipeline package

* Add basic tests for all exchange event entities

* Add tests for remaining entities

* Create separate test scripts in package.json and add new info to README

* Update db_setup.ts to revert migrations even if you are using docker

* Automatically pull the postgres image if needed

* Add comment about why NumberToBigIntTransformer is needed
2018-12-06 12:05:38 +01:00
Alex Browne
3ca876c574 [pipeline] Add additional documentation to the README (#1328) 2018-12-06 12:04:25 +01:00
Alex Browne
6d2f4b91a9 Reduce number of blocks per query in pull_missing_events 2018-12-06 12:04:25 +01:00
Jake Ellowitz
6739261a7e updating comment for 0x trusted tokens 2018-12-06 12:04:25 +01:00
Jake Ellowitz
72c45d9ea9 metadata and trusted sources in same raw table 2018-12-06 12:04:25 +01:00
Jake Ellowitz
75df8a0e8b Pull token metadata re trusted tokens 2018-12-06 12:04:25 +01:00
Alex Browne
f3c28afedd Add script for pulling missing block data 2018-12-06 12:04:25 +01:00
Alex Browne
80ab797d3a Check for special characters in table name in pull_missing_events 2018-12-06 12:04:25 +01:00
Alex Browne
d6dff5f86a Add support for pulling Cancel and CancelUpTo events 2018-12-06 12:04:25 +01:00
Alex Browne
4a715c30fd Add transactionHash to CancelUpToEvent and CancelEvent entities 2018-12-06 12:04:25 +01:00
Alex Browne
65bb8bdb5e Optimize database operations in pull_missing_events script 2018-12-06 12:04:25 +01:00
Alex Browne
5237e9c2ed Add homepage_url to relayers 2018-12-06 12:04:25 +01:00
Alex Browne
6bab2f70b9 Add workaround for broken save method 2018-12-06 12:04:25 +01:00
Alex Browne
a0a41b1b5d Fix chunk size in pull_missing_events 2018-12-06 12:04:25 +01:00
Alex Browne
f07dc5ae6d Use built-in chunk feature of TypeORM save method 2018-12-06 12:04:25 +01:00
Alex Browne
8cb6c2b51b Change some column types from varchar to numeric 2018-12-06 12:04:25 +01:00
Alex Browne
1f87bd8cf6 Add raw schema prefix to query in pull_missing_events 2018-12-06 12:04:25 +01:00
Alex Browne
a2aeca7b66 Rename taker and maker amounts in sra_orders 2018-12-06 12:04:25 +01:00
Alex Browne
e394c4fe1d Implement scraping sra orders from radar relay 2018-12-06 12:04:25 +01:00
Alex Browne
5c655eeda7 Preliminary work for adding RR order book scraping 2018-12-06 12:04:25 +01:00
Alex Browne
d932448563 Update schema for sra_orders 2018-12-06 12:04:25 +01:00
Jake Ellowitz
7f782b6af0 Pointing entities to raw schema
Fix linter issues
2018-12-06 12:04:25 +01:00
Jake Ellowitz
a3f4264790 Mapping dev schema to initial migration 2018-12-06 12:04:25 +01:00
Alex Browne
b818a62d74 Configure linter with --format stylish and fix linter errors 2018-12-06 12:04:25 +01:00
Alex Browne
e69cdfb7b5 Configure TypeORM for migrations. Add new package.json scripts. 2018-12-06 12:04:25 +01:00
Alex Browne
45da317e15 Rename Transaction.ts to transaction.ts
Rename Relayer.ts to relayer.ts

Rename Block.ts to block.ts
2018-12-06 12:04:25 +01:00
Alex Browne
c21cf55b86 Add better error handling for immediately invoked async functions 2018-12-06 12:04:25 +01:00
Alex Browne
2dfdc19c9e Rename table and column names 2018-12-06 12:04:25 +01:00
Alex Browne
012134001a Reorganize entities. Make scripts work from any directory. 2018-12-06 12:04:24 +01:00
Alex Browne
0f869b0545 Split index.ts into multiple scripts in scripts/ and detect last known block when pulling events 2018-12-06 12:04:24 +01:00
Alex Browne
774971c98c Remove unused function parameter in index.ts script 2018-12-06 12:04:24 +01:00
Alex Browne
25b6e7092f Use bigint for gasUsed and gasPrice in Transaction 2018-12-06 12:04:24 +01:00
Alex Browne
62412b8551 Update relayer code to use new relayer-registry format 2018-12-06 12:04:24 +01:00
Alex Browne
374d513090 Remove outdated info from README 2018-12-06 12:04:24 +01:00
Alex Browne
a72463dbdb Add build:ci to pipeline package.json 2018-12-06 12:04:24 +01:00
Alex Browne
b159d4e0ec Implement fetching and parsing relayer info 2018-12-06 12:04:24 +01:00
Alex Browne
1402a3dfae Implement support for getting and parsing blocks and transactions 2018-12-06 12:04:24 +01:00
Alex Browne
64865bc10f chore: Fix tslint config. 2018-12-06 12:04:24 +01:00
Alex Browne
ef00ac6f51 Fix bugs in merge_v2_events script and disable verbose logging 2018-12-06 12:04:24 +01:00
Alex Browne
7a7c66b0ec Enable verbose logging 2018-12-06 12:04:24 +01:00
Alex Browne
0166c29625 Rename index.ts -> merge_v2_events 2018-12-06 12:04:24 +01:00
Alex Browne
9c35d53694 Update script to work with existing v1 pipeline data 2018-12-06 12:04:24 +01:00
Alex Browne
8701f9a7b0 Update pipeline to use new @0x npm org for imports 2018-12-06 12:04:24 +01:00
Alex Browne
427c2cd164 Update to use ContractWrappers + Infura instead of Etherscan 2018-12-06 12:04:24 +01:00
Alex Browne
8c0dfc1936 Rebase and update to use new modularized artifacts 2018-12-06 12:04:24 +01:00
Alex Browne
392e89fcbb Add tests for SRA order conversion 2018-12-06 12:04:24 +01:00