diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..e7a3828 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,36 @@ +# Contributing guide + +Welcome to the Flashbots collective! We just ask you to be nice when you play with us. + +## Pre-commit + +We use pre-commit to maintain a consistent style, prevent errors, and ensure test coverage. + +To set up, install dependencies through `poetry`: + +``` +poetry install +``` + +Then install pre-commit hooks with: + +``` +poetry run pre-commit install +``` + +## Tests + +Run tests with: + +``` +kubectl exec deploy/mev-inspect-deployment -- poetry run pytest --cov=mev_inspect tests +``` + +## Send a pull request + +- Your proposed changes should be first described and discussed in an issue. +- Open the branch in a personal fork, not in the team repository. +- Every pull request should be small and represent a single change. If the problem is complicated, split it in multiple issues and pull requests. +- Every pull request should be covered by unit tests. + +We appreciate you, friend <3. diff --git a/README.md b/README.md index 221a5d5..318893b 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ # mev-inspect-py -> illuminating the dark forest 🌲💡 -**mev-inspect-py** is an MEV inspector for Ethereum +[![standard-readme compliant](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) +![Discord](https://img.shields.io/discord/755466764501909692) + +[Maximmal extractable value](https://ethereum.org/en/developers/docs/mev/) inspector for Ethereum, to illuminate the [dark forest](https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest/) 🌲💡 Given a block, mev-inspect finds: - miner payments (gas + coinbase) @@ -9,106 +11,118 @@ Given a block, mev-inspect finds: - swaps and [arbitrages](https://twitter.com/bertcmiller/status/1427632028263059462) - ...and more -Data is stored in Postgres for analysis +Data is stored in Postgres for analysis. -## Running locally -mev-inspect-py is built to run on kubernetes locally and in production +## Install -### Install dependencies +mev-inspect-py is built to run on kubernetes locally and in production. -First, setup a local kubernetes deployment - we use [Docker](https://www.docker.com/products/docker-desktop) and [kind](https://kind.sigs.k8s.io/docs/user/quick-start) +### Dependencies + +- [docker](https://www.docker.com/products/docker-desktop) +- [kind](https://kind.sigs.k8s.io/docs/user/quick-start) +- [kubectl](https://kubernetes.io/docs/tasks/tools/) +- [helm](https://helm.sh/docs/intro/install/) + +### Set up + +Ceate a new cluster with: -If using kind, create a new cluster with: ``` kind create cluster ``` -Next, install the kubernetes CLI [`kubectl`](https://kubernetes.io/docs/tasks/tools/) +Set an environment variable `RPC_URL` to an RPC for fetching blocks. -Then, install [helm](https://helm.sh/docs/intro/install/) - helm is a package manager for kubernetes - -Lastly, setup [Tilt](https://docs.tilt.dev/install.html) which manages running and updating kubernetes resources locally - -### Start up - -Set an environment variable `RPC_URL` to an RPC for fetching blocks Example: + ``` export RPC_URL="http://111.111.111.111:8546" ``` -**Note: mev-inspect-py currently requires an RPC with support for Erigon traces and receipts (not geth 😔)** +**Note: mev-inspect-py currently requires an RPC of a full archive node with support for Erigon traces and receipts (not geth 😔)** Next, start all services with: + ``` tilt up ``` -Press "space" to see a browser of the services starting up +Press "space" to see a browser of the services starting up. + +On first startup, you'll need to apply database migrations with: -On first startup, you'll need to apply database migrations. Apply with: ``` kubectl exec deploy/mev-inspect -- alembic upgrade head ``` -## Inspecting +## Usage ### Inspect a single block -Inspecting block [12914944](https://twitter.com/mevalphaleak/status/1420416437575901185) +Inspecting block [12914944](https://twitter.com/mevalphaleak/status/1420416437575901185): + ``` kubectl exec deploy/mev-inspect -- poetry run inspect-block 12914944 ``` ### Inspect many blocks -Inspecting blocks 12914944 to 12914954 +Inspecting blocks 12914944 to 12914954: + ``` kubectl exec deploy/mev-inspect -- poetry run inspect-many-blocks 12914944 12914954 ``` ### Inspect all incoming blocks -Start a block listener with +Start a block listener with: + ``` kubectl exec deploy/mev-inspect -- /app/listener start ``` By default, it will pick up wherever you left off. -If running for the first time, listener starts at the latest block +If running for the first time, listener starts at the latest block. + +See logs for the listener with: -See logs for the listener with ``` kubectl exec deploy/mev-inspect -- tail -f listener.log ``` -And stop the listener with +And stop the listener with: + ``` kubectl exec deploy/mev-inspect -- /app/listener stop ``` -## Exploring +### Exploring All inspect output data is stored in Postgres. To connect to the local Postgres database for querying, launch a client container with: + ``` kubectl run -i --rm --tty postgres-client --env="PGPASSWORD=password" --image=jbergknoff/postgresql-client -- mev_inspect --host=postgresql --user=postgres ``` -When you see the prompt +When you see the prompt: + ``` mev_inspect=# ``` You're ready to query! -Try finding the total number of swaps decoded with UniswapV3Pool +Try finding the total number of swaps decoded with UniswapV3Pool: + ``` SELECT COUNT(*) FROM swaps WHERE abi_name='UniswapV3Pool'; ``` -or top 10 arbs by gross profit that took profit in WETH +or top 10 arbs by gross profit that took profit in WETH: + ``` SELECT * FROM arbitrages @@ -117,78 +131,81 @@ ORDER BY profit_amount DESC LIMIT 10; ``` -Postgres tip: Enter `\x` to enter "Explanded display" mode which looks nicer for results with many columns - -## Contributing - -### Guide - -✨ Coming soon - -### Pre-commit - -We use pre-commit to maintain a consistent style, prevent errors, and ensure test coverage. - -To set up, install dependencies through poetry -``` -poetry install -``` - -Then install pre-commit hooks with -``` -poetry run pre-commit install -``` - -### Tests - -Run tests with -``` -kubectl exec deploy/mev-inspect -- poetry run pytest --cov=mev_inspect tests -``` +Postgres tip: Enter `\x` to enter "Explanded display" mode which looks nicer for results with many columns. ## FAQ ### How do I delete / reset my local postgres data? -Stop the system if running +Stop the system if running: + ``` tilt down ``` -Delete it with +Delete it with: + ``` kubectl delete pvc data-postgresql-postgresql-0 ``` -Start back up again +Start back up again: + ``` tilt up ``` -And rerun migrations to create the tables again +And rerun migrations to create the tables again: + ``` kubectl exec deploy/mev-inspect -- alembic upgrade head ``` ### I was using the docker-compose setup and want to switch to kube, now what? -Re-add the old `docker-compose.yml` file to your mev-inspect-py directory +Re-add the old `docker-compose.yml` file to your mev-inspect-py directory. A copy can be found [here](https://github.com/flashbots/mev-inspect-py/blob/ef60c097719629a7d2dc56c6e6c9a100fb706f76/docker-compose.yml) -Tear down docker-compose resources +Tear down docker-compose resources: + ``` docker compose down ``` -Then go through the steps in the current README for kube setup +Then go through the steps in the current README for kube setup. ### Error from server (AlreadyExists): pods "postgres-client" already exists -This means the postgres client container didn't shut down correctly -Delete this one with +This means the postgres client container didn't shut down correctly. + +Delete this one with: + ``` kubectl delete pod/postgres-client ``` -Then start it back up again +Then start it back up again. + +## Maintainers + +- [@lukevs](https://github.com/lukevs) + +## Contributing + +[Flashbots](https://flashbots.net) is a research and development collective working on mitigating the negative externalities of decentralized economies. We contribute with the larger free software community to illuminate the dark forest. + +You are welcome here <3. + +- If you want to join us, come and say hi in our [Discord chat](https://discord.gg/7hvTycdNcK). +- If you have a question, feedback or a bug report for this project, please [open a new Issue](https://github.com/flashbots/flashbots-repository-template/issues). +- If you would like to contribute with code, check the [CONTRIBUTING file](CONTRIBUTING.md). +- We just ask you to be nice. + +## Security + +If you find a security vulnerability on this project or any other initiative related to Flashbots, please let us know sending an email to security@flashbots.net. + +--- + +Made with ☀️ by the ⚡🤖 collective.