crowetic/protocol
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,381 Commits 83 Branches 5,211 Tags
90ad681a9e82a65d8dd63a9236173f975b86fec1
Commit Graph

171 Commits

Author SHA1 Message Date
xianny
8b3b4d983f rename variable and define default in only 1 location 2018-12-07 09:43:55 -08:00
xianny
ac971685b3 upgrade throttling code 2018-12-07 09:43:55 -08:00
Xianny
78d0ab1aa2 Fix/pipeline/ohlcv (#1393) 
The OHLCV script in data pipeline quits early when we get no data from Crypto Compare. Sometimes Crypto Compare gives us a valid empty response (e.g. when we query for way back in time) and we need to just continue. This adds better filtering for the types of Crypto Compare responses to detect when we should continue and when we should really quit.
 2018-12-05 16:05:06 -08:00
Alex Browne
08eb0b91b6 Fix RadarRelay timestamps (#1391) 
* Fixing rr timestamps

* Apply prettier
 2018-12-05 12:27:32 -08:00
Alex Browne
e0348f9c04 Change type of transactions.gas_used and gas_price to BigNumber/numeric 2018-12-04 20:20:49 -08:00
Alex Browne
2e704ac01a Fix prettier 2018-12-04 20:08:32 -08:00
Alex Browne
00f86ca0f7 Address PR feedback 2018-12-04 20:04:08 -08:00
Fabio B
549f5e4655 Use a string template in packages/pipeline/src/scripts/pull_missing_events.ts 
Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
 2018-12-04 19:56:23 -08:00
Fabio B
ebdc539da7 Fix typo in packages/pipeline/src/parsers/bloxy/index.ts 
Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
 2018-12-04 19:31:19 -08:00
Alex Browne
8721d4ed7a Fix linter 2018-12-04 15:23:15 -08:00
Xianny
8c21a700ba pull OHLCV records from Crypto Compare (#1349) 
* [WIP] pull OHLCV records from Crypto Compare

* lint

* refactor to pull logic out of script and into modules

* add entity test for ohlcv_external entity

* implement rate limit and chronological backfill for ohlcv

* add unit tests; cleanup variable names

* Fetch OHLCV pairs params from events table

* better method names

* fix outdated test

* lint

* Clean up after review

* oops

* fix failing test

* better filtering of most recent records

* fix bug when generating pairs

* fix default earliest backfill date

* fix bug with retrieving backfill time

* prettier
 2018-12-04 14:26:03 -08:00
zkao
87ffa5d7ab Token_orderbook_snapshots for Ddex and Paradex(#1354) 
* Implements the TokenOrderbookSnapshot Table

* Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
 2018-12-04 14:25:42 -08:00
Alex Browne
7198b441e0 Add script for parsing competing dex trades from Bloxy (#1355) 2018-12-04 14:25:42 -08:00
Alex Browne
3d211c415b Introduce framework for running basic tests for entities (#1344) 
* Introduce framework for running basic tests for entities

* Add pipeline tests to CircleCI config

* Make pipeline tests more configurable and fix CircleCI config

* Add coverage dir to pipeline package

* Add basic tests for all exchange event entities

* Add tests for remaining entities

* Create separate test scripts in package.json and add new info to README

* Update db_setup.ts to revert migrations even if you are using docker

* Automatically pull the postgres image if needed

* Add comment about why NumberToBigIntTransformer is needed
 2018-12-04 14:25:38 -08:00
Alex Browne
96fdb9b766 Reduce number of blocks per query in pull_missing_events 2018-12-04 14:24:48 -08:00
Jake Ellowitz
1aa3f9d69f updating comment for 0x trusted tokens 2018-12-04 14:24:48 -08:00
Jake Ellowitz
dea89c4e22 metadata and trusted sources in same raw table 2018-12-04 14:24:48 -08:00
Jake Ellowitz
c6af5131b0 Pull token metadata re trusted tokens 2018-12-04 14:24:48 -08:00
Alex Browne
9986717671 Add script for pulling missing block data 2018-12-04 14:24:48 -08:00
Alex Browne
5cad2ad174 Check for special characters in table name in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
24fd2d9730 Add support for pulling Cancel and CancelUpTo events 2018-12-04 14:24:48 -08:00
Alex Browne
0397ff8b22 Add transactionHash to CancelUpToEvent and CancelEvent entities 2018-12-04 14:24:48 -08:00
Alex Browne
1a25862086 Optimize database operations in pull_missing_events script 2018-12-04 14:24:48 -08:00
Alex Browne
b771020f88 Add homepage_url to relayers 2018-12-04 14:24:48 -08:00
Alex Browne
b42cf0c797 Add workaround for broken save method 2018-12-04 14:24:48 -08:00
Alex Browne
8076333795 Fix chunk size in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
b0a2c10e11 Use built-in chunk feature of TypeORM save method 2018-12-04 14:24:48 -08:00
Alex Browne
303bbc42f4 Change some column types from varchar to numeric 2018-12-04 14:24:48 -08:00
Alex Browne
10e93bb01f Add raw schema prefix to query in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
a948305e7a Rename taker and maker amounts in sra_orders 2018-12-04 14:24:48 -08:00
Alex Browne
26280e4aba Implement scraping sra orders from radar relay 2018-12-04 14:24:48 -08:00
Alex Browne
55bbe1954b Preliminary work for adding RR order book scraping 2018-12-04 14:24:48 -08:00
Alex Browne
2cbb82eb04 Update schema for sra_orders 2018-12-04 14:24:48 -08:00
Jake Ellowitz
96134003e1 Pointing entities to raw schema 
Fix linter issues
 2018-12-04 14:25:00 -08:00
Jake Ellowitz
599a3f9b96 Mapping dev schema to initial migration 2018-12-04 14:24:48 -08:00
Alex Browne
688d277b30 Configure linter with --format stylish and fix linter errors 2018-12-04 14:24:48 -08:00
Alex Browne
329c68f610 Configure TypeORM for migrations. Add new package.json scripts. 2018-12-04 14:24:48 -08:00
Alex Browne
6fb333f200 Rename Transaction.ts to transaction.ts 
Rename Relayer.ts to relayer.ts

Rename Block.ts to block.ts
 2018-12-04 14:24:55 -08:00
Alex Browne
410a924495 Add better error handling for immediately invoked async functions 2018-12-04 14:24:48 -08:00
Alex Browne
53cc9e9bed Rename table and column names 2018-12-04 14:24:48 -08:00
Alex Browne
ccad046eb6 Reorganize entities. Make scripts work from any directory. 2018-12-04 14:24:48 -08:00
Alex Browne
954c3b9272 Split index.ts into multiple scripts in scripts/ and detect last known block when pulling events 2018-12-04 14:24:48 -08:00
Alex Browne
9cb89725c9 Remove unused function parameter in index.ts script 2018-12-04 14:24:48 -08:00
Alex Browne
830d0f3b21 Use bigint for gasUsed and gasPrice in Transaction 2018-12-04 14:24:48 -08:00
Alex Browne
8248fbb231 Update relayer code to use new relayer-registry format 2018-12-04 14:24:48 -08:00
Alex Browne
5e22a862b7 Implement fetching and parsing relayer info 2018-12-04 14:24:48 -08:00
Alex Browne
bb440b683a Implement support for getting and parsing blocks and transactions 2018-12-04 14:24:48 -08:00
Alex Browne
d959b3e234 chore: Fix tslint config. 2018-12-04 14:24:49 -08:00
Alex Browne
aeff948c9a Fix bugs in merge_v2_events script and disable verbose logging 2018-12-04 14:24:48 -08:00
Alex Browne
7945d2ea62 Enable verbose logging 2018-12-04 14:24:48 -08:00