Skip to content

Commit

Permalink
[BLOCK-2260] merge upstream changes for ultra firehose antelope (#10)
Browse files Browse the repository at this point in the history
* fix decoding panic by double checking variant type bounds (pinax-network#97)

* fix decoding panic by double checking variant type bounds

* remove eos-go replacement

* update firehose-core to v0.2.1 (pinax-network#99)

* bump firehose-core to v0.2.1

* implement new firecore.Block methods

* cleanup

* add test

* bump firehose-core to v0.2.2 (pinax-network#100)

* add buf yaml (pinax-network#101)

* bump firehose-core to v0.2.3 (pinax-network#102)

* bump firehose-core to v0.2.4 (pinax-network#104)

* remove unused proto dependency on generate (pinax-network#106)

* upgrade to firehose-core v1.0.0 (pinax-network#107)

* start work upgrading to firehose-core v1

* fix testing types

* remove tools

* fix test build

* cleanup

* cleanup

* cleanup

* add Silkworm poller for Antelope EVM (pinax-network#108)

* add Silkworm poller for Antelope EVM

* remove config.yaml

* Bump github.com/libp2p/go-libp2p from 0.26.3 to 0.27.8 (pinax-network#109)

Bumps [github.com/libp2p/go-libp2p](https://github.com/libp2p/go-libp2p) from 0.26.3 to 0.27.8.
- [Release notes](https://github.com/libp2p/go-libp2p/releases)
- [Changelog](https://github.com/libp2p/go-libp2p/blob/master/CHANGELOG.md)
- [Commits](libp2p/go-libp2p@v0.26.3...v0.27.8)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add workaround for missing finalized query on eos evm (pinax-network#110)

* enable multitests for the consolereader (pinax-network#111)

* Bump golang.org/x/crypto from 0.14.0 to 0.17.0 (pinax-network#112)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](golang/crypto@v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add check-blocks tool to check merged blocks for decoding issues (pinax-network#114)

* fix check-blocks starting at block 0 (pinax-network#115)

* fix check-blocks starting at block 0

* remove log

* add block sanitize function for comparisons (pinax-network#116)

* add leap 5.0 test data (pinax-network#117)

* bump firehose-core to v1.1.0 (pinax-network#118)

* bump firehose-core to v1.1.1 (pinax-network#119)

* use a constant timestamp for sanitizing blocks (pinax-network#120)

* add missing sanitation for failed deferred transaction traces (pinax-network#121)

* improve the block sanitation

* remove accidentally removed sanitize

* bump firehose-core to v1.2.1 (pinax-network#122)

* bump firehose-core to v1.2.1

* fix sanitizeBlockForCompare, remove block poller

* run go mod tidy

* update Readme (pinax-network#123)

* update Readme

* fix typo

* add support and subgraph sections

* fix typos (pinax-network#124)

* bump firehose-core to v1.2.2 (pinax-network#125)

* bump firehose-core to v1.2.3 (pinax-network#126)

* bump firehose-core to v1.2.4 (pinax-network#127)

* bump firehose-core to v1.2.5 (pinax-network#128)

* Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (pinax-network#129)

Bumps google.golang.org/protobuf from 1.32.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump firehose-core to v1.3.1 (pinax-network#130)

* bump firehose-core to v1.3.2 (pinax-network#131)

* bump firehose-core to v1.3.3 (pinax-network#132)

* bump firehose-core to v1.3.3

* run go mod tidy

* [BLOCK-2260] Remove obsolete files

* [BLOCK-2260] Remove another obsolete files

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Frederik Schöll <[email protected]>
Co-authored-by: Yaro Shkvorets <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  • Loading branch information
4 people authored Apr 10, 2024
1 parent 8e79613 commit 91f02db
Show file tree
Hide file tree
Showing 11 changed files with 254 additions and 478 deletions.
57 changes: 48 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,57 @@
# Firehose on Antelope

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

This is Streamingfast's [Firehose](https://firehose.streamingfast.io) implementation adapted for Antelope chains,
effectively enabling [Substreams](https://substreams.streamingfast.io) on all Antelope chains.
This is the Antelope chain-specific implementation part of firehose-core and enables both
[Firehose](https://firehose.streamingfast.io/introduction/firehose-overview)
and [Substreams](https://substreams.streamingfast.io) on Antelope chains with native blocks.

## For Developers

To get started with Firehose and Substreams, you need to sign up on https://app.pinax.network to get yourself an api
key. You'll also find quickstarts there to get you started and all of our available endpoints (we currently provide both
Firehose and Substreams endpoints for EOS, WAX and Telos, as well as different testnets).

For connecting to **Firehose** endpoints, you'll need the Protobufs which are published on
[buf.build](https://buf.build/pinax/firehose-antelope/docs/main). Some Golang example code on how to set up a Firehose
client can be found [here](https://github.com/pinax-network/firehose-examples-go).

To **consume** Antelope Substreams, please have a look at the
[documentation](https://substreams.streamingfast.io/documentation/consume). You can also find Substreams to deploy in
our Substreams repository [here](https://github.com/pinax-network/substreams) and on
[substreams.dev](https://substreams.dev).

To **develop** Antelope Substreams, have a look at
the [documentation](https://substreams.streamingfast.io/documentation/develop) here and at the Pinax SDK for Antelope
Substreams which can be found [here](https://github.com/pinax-network/substreams-antelope).

A collection of resources around Substreams can also be found
on [Awesome Substreams](https://github.com/pinax-network/awesome-substreams).

### Subgraphs

Although The Graph does not officially support any Antelope chains yet, it's possible to write Subgraphs based on
Substreams and set up your own Graph node. You can find an
example [here](https://github.com/pinax-network/eosio.token-subgraph) on how to achieve this.

## For Operators

Please have a look at the documentation [here](https://firehose.streamingfast.io) on how to set up your own Firehose &
Substreams stack. Note that indexing larger Antelope chains such as EOS or WAX requires parallel processing of the chain
and a lot of resources to have the indexing done in a reasonable time frame.

# Usage
### EOS EVM

## Release
This implementation provides native Antelope blocks, including all Antelope specific block data. In case you are looking
for operating Firehose & Substreams for EOS EVM, please have a look at
the [firehose-ethereum](https://github.com/streamingfast/firehose-ethereum) repository; it provides a generic evm poller
to poll the EVM blocks from an RPC node.

Use the `./bin/release.sh` Bash script to perform a new release. It will ask you questions
as well as driving all the required commands, performing the necessary operation automatically.
The Bash script runs in dry-mode by default, so you can check first that everything is all right.
## Support

Releases are performed using [goreleaser](https://goreleaser.com/).
In case of any questions around the Pinax endpoints or technology, feel free to hit us on
our [Discord server](https://discord.gg/pinax). For more generic questions around Substreams, you might also find the
[Streamingfast Discord server](https://discord.gg/jZwqxJAvRs) useful.

## To run `fireantelope` and `dfuseeos`

Expand Down Expand Up @@ -74,7 +113,7 @@ dfuseeos start -c ./devel/standard/dfuse.yaml

- If not work try to update `firehose-data/reader/config/config.ini`, change `p2p-peer-address = 127.0.0.1:9876` to `p2p-peer-address = host.docker.internal:9876`

# Deployment
## Deployment

Todo: describe deployments for firehose and substreams

Expand Down
123 changes: 0 additions & 123 deletions blockfetcher/rpc.go

This file was deleted.

29 changes: 0 additions & 29 deletions blockfetcher/silkworm.go

This file was deleted.

22 changes: 0 additions & 22 deletions cmd/fireantelope/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,6 @@ func Chain() *firecore.Chain[*pbantelope.Block] {

BlockFactory: func() firecore.Block { return new(pbantelope.Block) },

//BlockIndexerFactories: map[string]firecore.BlockIndexerFactory[*pbantelope.Block]{
// transform.ReceiptAddressIndexShortName: transform.NewNearBlockIndexer,
//},
//
//BlockTransformerFactories: map[protoreflect.FullName]firecore.BlockTransformerFactory{
// transform.HeaderOnlyMessageName: transform.NewHeaderOnlyTransformFactory,
// transform.ReceiptFilterMessageName: transform.BasicReceiptFilterFactory,
//},

ConsoleReaderFactory: codec.NewConsoleReader,

RegisterExtraStartFlags: func(flags *pflag.FlagSet) {
Expand All @@ -51,28 +42,15 @@ func Chain() *firecore.Chain[*pbantelope.Block] {
flags.Bool("reader-node-overwrite-node-files", false, "Force download of node-key and config files even if they already exist on the machine.")
},

// ReaderNodeBootstrapperFactory: newReaderNodeBootstrapper,

Tools: &firecore.ToolsConfig[*pbantelope.Block]{

RegisterExtraCmd: func(chain *firecore.Chain[*pbantelope.Block], parent *cobra.Command, zlog *zap.Logger, tracer logging.Tracer) error {
//toolsCmd.AddCommand(newToolsGenerateNodeKeyCmd(chain))
//toolsCmd.AddCommand(newToolsBackfillCmd(zlog))
parent.AddCommand(newPollerCmd(zlog, tracer))
parent.AddCommand(newSilkwormPollerCmd(zlog, tracer))
parent.AddCommand(newCheckBlocksCmd(zlog))

return nil
},

SanitizeBlockForCompare: sanitizeBlockForCompare,

//TransformFlags: map[string]*firecore.TransformFlag{
// "receipt-account-filters": {
// Description: "Comma-separated accounts to use as filter/index. If it contains a colon (:), it will be interpreted as <prefix>:<suffix> (each of which can be empty, ex: 'hello:' or ':world')",
// Parser: parseReceiptAccountFilters,
// },
//},
},
}

Expand Down
85 changes: 0 additions & 85 deletions cmd/fireantelope/poller.go

This file was deleted.

File renamed without changes.
Loading

0 comments on commit 91f02db

Please sign in to comment.