diff --git a/README.md b/README.md index bef5711..9e1f588 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ kubectl exec deploy/mev-inspect -- alembic upgrade head Inspecting block [12914944](https://twitter.com/mevalphaleak/status/1420416437575901185): ``` -kubectl exec deploy/mev-inspect -- poetry run inspect-block 12914944 +./mev inspect 12914944 ``` ### Inspect many blocks @@ -72,7 +72,7 @@ kubectl exec deploy/mev-inspect -- poetry run inspect-block 12914944 Inspecting blocks 12914944 to 12914954: ``` -kubectl exec deploy/mev-inspect -- poetry run inspect-many-blocks 12914944 12914954 +./mev inspect-many 12914944 12914954 ``` ### Inspect all incoming blocks @@ -80,24 +80,46 @@ kubectl exec deploy/mev-inspect -- poetry run inspect-many-blocks 12914944 12914 Start a block listener with: ``` -kubectl exec deploy/mev-inspect -- /app/listener start +./mev listener start ``` By default, it will pick up wherever you left off. If running for the first time, listener starts at the latest block. -See logs for the listener with: +Tail logs for the listener with: ``` -kubectl exec deploy/mev-inspect -- tail -f listener.log +./mev listener tail ``` And stop the listener with: ``` -kubectl exec deploy/mev-inspect -- /app/listener stop +./mev listener stop ``` +### Backfilling + +For larger backfills, you can inspect many blocks in parallel using kubernetes + +To inspect blocks 12914944 to 12915044 divided across 10 worker pods: +``` +./mev backfill 12914944 12915044 10 +``` + +You can see worker pods spin up then complete by watching the status of all pods +``` +watch kubectl get pods +``` + +To watch the logs for a given pod, take its pod name using the above, then run: +``` +kubectl logs -f pod/mev-inspect-backfill-abcdefg +``` + +(where `mev-inspect-backfill-abcdefg` is your actual pod name) + + ### Exploring All inspect output data is stored in Postgres. @@ -105,7 +127,7 @@ All inspect output data is stored in Postgres. To connect to the local Postgres database for querying, launch a client container with: ``` -kubectl run -i --rm --tty postgres-client --env="PGPASSWORD=password" --image=jbergknoff/postgresql-client -- mev_inspect --host=postgresql --user=postgres +./mev db ``` When you see the prompt: diff --git a/listener b/listener index 30ebede..0a86c50 100755 --- a/listener +++ b/listener @@ -25,6 +25,9 @@ case "$1" in start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE echo "." ;; + tail) + tail -f listener.log + ;; restart) echo -n "Restarting daemon: "$NAME start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE @@ -40,7 +43,7 @@ case "$1" in ;; *) - echo "Usage: "$1" {start|stop|restart}" + echo "Usage: "$1" {start|stop|restart|tail}" exit 1 esac diff --git a/mev b/mev index a936bf1..2988390 100755 --- a/mev +++ b/mev @@ -24,6 +24,9 @@ case "$1" in echo "Connecting to $DB_NAME" db ;; + listener) + ./listener $2 + ;; backfill) start_block_number=$2 end_block_number=$3 @@ -37,12 +40,19 @@ case "$1" in echo "Inspecting block $block_number" kubectl exec -ti deploy/mev-inspect -- poetry run inspect-block $block_number ;; + inspect-many) + start_block_number=$2 + end_block_number=$3 + echo "Inspecting from block $start_block_number to $end_block_number" + kubectl exec -ti deploy/mev-inspect -- \ + poetry run inspect-many-blocks $start_block_number $end_block_number + ;; test) echo "Running tests" kubectl exec -ti deploy/mev-inspect -- poetry run pytest tests ;; *) - echo "Usage: "$1" {inspect|test}" + echo "Usage: "$1" {db|backfill|inspect|test}" exit 1 esac