crowetic/protocol
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,828 Commits 83 Branches 5,211 Tags
8885f543aeddee3b27e77329326fcfb27ed24025
Commit Graph

57 Commits

Author SHA1 Message Date
askeluv
8885f543ae Pull exchange contract transactions from Etherscan 2019-02-21 20:37:55 +08:00
Francesco Agosti
39390865da Merge pull request #1612 from 0xProject/feature/pipeline/add-radar 
[pipeline] Add Radar orders and maker_address column to token_orderbook_snapshots
 2019-02-20 14:17:42 -08:00
fragosti
4952cd5afb Factor out redundant source param in parse order functions 2019-02-20 13:45:14 -08:00
Alex Svanevik
154e7a6a8a Merge pull request #1601 from askeluv/feature/pipeline/edps-slippage 
Slippage calculations from Ethereum DEX Prices Service (EDPS)
 2019-02-20 22:51:23 +08:00
F. Eugene Aumson
1232a9a03d Integrate one-time dump and API for nonfungible.com (#1603) 
* Add script for pulling NFT trade data from nonfungible.com

* corrections for current state of API

* change data model to match data source

* change primary key

* pull data from initial dump first, then API

* pull all supported NFT's, not just cryptokitties

* disable problematic data sources

* rename function to satisfy linter

* Rename table to nonfungible_dot_com_trades

* rename parser module to nonfungible_dot_com from non_fungible_dot_com, for consistency

* correct mistaken reference to Bloxy

* rename NonfungibleDotComTrade to ...TradeResponse

* `NftTrade` -> `NonfungibleDotComTrade`

* rename files to match prior object renaming

* use fetchAsync instead of axios

* improve fetchAsync error message: include URL

* avoid non-null contraints in API trades too, not just for trades from the one-time dump

* disable mythereum publisher
 2019-02-19 19:07:42 -08:00
askeluv
379e828fc1 Moved calculateSlippage from parsers to transformers 2019-02-20 08:55:21 +08:00
askeluv
0e644da6ac Fixed linting issues 2019-02-14 17:39:12 +08:00
askeluv
ab286d851d Changed Maps to EdpsWrappers 2019-02-14 15:30:58 +08:00
askeluv
c36e1ad056 Changes made as per comments - still have two minor things to fix 2019-02-13 20:23:07 +08:00
fragosti
b7eb2e4746 Merge branch 'development' of https://github.com/0xProject/0x-monorepo into feature/pipeline/add-radar 2019-02-12 14:56:02 -08:00
fragosti
4db9b8b0e3 Add script for pulling radar orderbook 2019-02-12 14:49:32 -08:00
askeluv
82dffe9d0e Added remaining tests + fixed linting issues 2019-02-12 19:34:05 +08:00
askeluv
c0a2f429b6 Added saving to database 2019-02-12 12:30:10 +08:00
F. Eugene Aumson
629a8d6328 Add migration to fix exchange events primary keys (#1593) 
* Add migration to fix exchange events primary key

* correct comment: "foreign key" -> "primary key"

* Refine hack to handle only the expected error

* Add tx hash to erc20 approval events primary key
 2019-02-08 13:21:41 -05:00
askeluv
7ad4cb0078 [WIP] Slippage calculations from Ethereum DEX Prices Service (EDPS) 2019-02-08 18:54:38 +08:00
F. Eugene Aumson
c859788fde Replace console.log with logUtils.log (#1515) 2019-01-14 18:21:12 -05:00
Alex Browne
89e398fa39 Update prettier to version ^1.15.3 2019-01-10 17:41:13 -08:00
Xianny
27fc640a9e fetch and save copper (#1472) 
Fetch and save Copper CRM
 2019-01-08 13:50:51 -08:00
Alex Browne
7dda953bc9 Optimize SQL queries in pull_missing_blocks (#1458) 
* Optimize SQL queries in pull_missing_blocks

* Update comment in pull_missing_blocks
 2019-01-07 15:04:00 -08:00
Alex Browne
44d9cc53b8 Fix bug in pull_missing_blocks with incorrect start block (#1438) 2018-12-17 17:07:22 -08:00
Alex Browne
28713bdb38 Pull approval events for ZRX and DAI (#1430) 2018-12-17 11:36:03 -08:00
Leonid Logvinov
1534505652 Fix linter errors 2018-12-12 15:32:41 -08:00
Alex Browne
5cff911035 Make pull_missing_blocks script consider all events with block numbers (#1420) 2018-12-11 17:31:16 -08:00
zkao
42be1a429f track idex orderbook snapshots (#1397) 
* Track Idex and Oasis Orderbook Snapshots
 2018-12-11 15:48:54 -08:00
Alex Browne
b4cdb14b9b Refactor event scraping and add support for scraping ERC20 approval events (#1401) 
* Refactor event scraping and add support for scraping ERC20 approval events

* Add tests for data_sources/contract-wrappers/utils
 2018-12-11 15:16:05 -08:00
xianny
096c4c8f2b change to camelCase 2018-12-10 10:57:36 -08:00
xianny
8b3b4d983f rename variable and define default in only 1 location 2018-12-07 09:43:55 -08:00
Xianny
78d0ab1aa2 Fix/pipeline/ohlcv (#1393) 
The OHLCV script in data pipeline quits early when we get no data from Crypto Compare. Sometimes Crypto Compare gives us a valid empty response (e.g. when we query for way back in time) and we need to just continue. This adds better filtering for the types of Crypto Compare responses to detect when we should continue and when we should really quit.
 2018-12-05 16:05:06 -08:00
Alex Browne
08eb0b91b6 Fix RadarRelay timestamps (#1391) 
* Fixing rr timestamps

* Apply prettier
 2018-12-05 12:27:32 -08:00
Alex Browne
2e704ac01a Fix prettier 2018-12-04 20:08:32 -08:00
Alex Browne
00f86ca0f7 Address PR feedback 2018-12-04 20:04:08 -08:00
Fabio B
549f5e4655 Use a string template in packages/pipeline/src/scripts/pull_missing_events.ts 
Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
 2018-12-04 19:56:23 -08:00
Alex Browne
8721d4ed7a Fix linter 2018-12-04 15:23:15 -08:00
Xianny
8c21a700ba pull OHLCV records from Crypto Compare (#1349) 
* [WIP] pull OHLCV records from Crypto Compare

* lint

* refactor to pull logic out of script and into modules

* add entity test for ohlcv_external entity

* implement rate limit and chronological backfill for ohlcv

* add unit tests; cleanup variable names

* Fetch OHLCV pairs params from events table

* better method names

* fix outdated test

* lint

* Clean up after review

* oops

* fix failing test

* better filtering of most recent records

* fix bug when generating pairs

* fix default earliest backfill date

* fix bug with retrieving backfill time

* prettier
 2018-12-04 14:26:03 -08:00
zkao
87ffa5d7ab Token_orderbook_snapshots for Ddex and Paradex(#1354) 
* Implements the TokenOrderbookSnapshot Table

* Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
 2018-12-04 14:25:42 -08:00
Alex Browne
7198b441e0 Add script for parsing competing dex trades from Bloxy (#1355) 2018-12-04 14:25:42 -08:00
Jake Ellowitz
1aa3f9d69f updating comment for 0x trusted tokens 2018-12-04 14:24:48 -08:00
Jake Ellowitz
dea89c4e22 metadata and trusted sources in same raw table 2018-12-04 14:24:48 -08:00
Jake Ellowitz
c6af5131b0 Pull token metadata re trusted tokens 2018-12-04 14:24:48 -08:00
Alex Browne
9986717671 Add script for pulling missing block data 2018-12-04 14:24:48 -08:00
Alex Browne
5cad2ad174 Check for special characters in table name in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
24fd2d9730 Add support for pulling Cancel and CancelUpTo events 2018-12-04 14:24:48 -08:00
Alex Browne
1a25862086 Optimize database operations in pull_missing_events script 2018-12-04 14:24:48 -08:00
Alex Browne
b42cf0c797 Add workaround for broken save method 2018-12-04 14:24:48 -08:00
Alex Browne
8076333795 Fix chunk size in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
b0a2c10e11 Use built-in chunk feature of TypeORM save method 2018-12-04 14:24:48 -08:00
Alex Browne
303bbc42f4 Change some column types from varchar to numeric 2018-12-04 14:24:48 -08:00
Alex Browne
10e93bb01f Add raw schema prefix to query in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
26280e4aba Implement scraping sra orders from radar relay 2018-12-04 14:24:48 -08:00
Alex Browne
55bbe1954b Preliminary work for adding RR order book scraping 2018-12-04 14:24:48 -08:00