Our development journey with Tezos began in 2019, when we released the first version of TzStats — just on time to replace TZscan. From day one, our goal at Blockwatch was clear: help blockchain ecosystems thrive by providing transparent and reliable data. Our dedication to the Tezos community has not only persisted but deepened, and our latest offering, TzPro, stands as a testament to this commitment: a comprehensive API service delivering unrivaled RPC, DeFi and NFT data.

We're grateful for the support we've received and what we've learned, and we are proud to have made a productive impact. Now, we're entering a new chapter. Read on for a recap and a future outlook.

What we achieved

Throughout the years we've built and maintained a stack of software and services for Tezos, most notably:

• TzStats block explorer
• TzIndex, all-in-one zero-conf blockchain indexer for Tezos
• TzGo, low-level Tezos Go SDK for reliable, high-performance applications
• TzCompose, an automation tool to setup test cases and deploy complex contract ecosystems
• TzGen, a code generator to produce pure Go clients for smart contract interfaces
• TzPro, a comprehensive API service for Tezos data, from low-level RPCs to indexed L1, token, DeFi, NFT and wallet identity data
• Tezos Polywrap integration, making it easy to use Tezos in any application written in any language

Charting a new course

At Blockwatch, our roots have always been deeply embedded in data engineering. This is what we do best and enjoy the most. With excitement, we announce our new venture: building a planet-scale P2P data warehouse. This bold step is not just a pivot but a progression of our foundational mission to harness data's full potential.

As part of this shift, we're also ensuring that our legacy with Tezos is well preserved. We are collaborating with Tezos Foundation and Trilitech to transition the maintenance and operations of TzStats and TzPro. This move guarantees that these critical components of the Tezos infrastructure will continue to be available and evolve under capable stewardship.

Welcome to the future

Our bond with the Tezos community remains unbroken. We're excited about the possibilities that lie ahead and are eager to explore new ways to support and collaborate with many of the great individuals and organizations we've come to know through Tezos.

We extend our gratitude to everyone involved in the Tezos ecosystem. Your passion and innovation have been a continuous source of inspiration. As we look to the future, we carry forward the spirit of commitment to excellence that has defined our journey thus far. Here's to continued growth and prosperity for all in the Tezos community and beyond!

Keep building,
Team Blockwatch

At Blockwatch, we've continuously been building and maintaining infrastructure and tooling for Tezos since the early betanet days in mid 2018. Today, the Blockwatch Tezos Stack is an invaluable asset to any developer building on or interacting with Tezos. In this post, I'm excited to guide you through and assist you in uncovering the stack's ability to empower you.

• TzGo SDK: low-level Tezos Go SDK focused on correctness, stability, and compliance with the Tezos protocol. Repository: https://github.com/blockwatch-cc/tzgo.
• TzCompose: Tezos automation and testing framework, part of TzGo. Repository: https://github.com/blockwatch-cc/tzgo/tree/master/cmd/tzcompose.
• TzIndex: all-in-one zero-conf Tezos indexer. Repository: https://github.com/blockwatch-cc/tzindex. Available as PRO version.
• TzPro: Tezos infrastructure for developers. Powerful APIs and SDKs. Platform: https://tzpro.io.

TzGo: Tezos Go SDK

TzGo is our official Tezos Go SDK. It's designed with a focus on correctness, stability, and compliance with the Tezos protocol. The SDK is free to use in commercial and non-commercial projects with a permissive license. It supports binary and JSON encodings for all Tezos types including Micheline smart contract data and all transaction formats.

TzGo is perfectly suited for high-performance apps and services that read from and write to the Tezos blockchain. The library is structured into packages that provide multiple layers of building blocks for maximum freedom of choice. All types and interfaces are engineered to be easily composable. Application programmers can directly use low-level types for greater control of details and communication patterns or use more convenient high-level functionality which hides details and allows the sending of several RPC calls in the background.

Currently, TzGo is compatible with all Tezos protocols up to v17 Nairobi.

TzGo Core Packages

• Low-level Types Library (to handle hashes, addresses, keys, signatures, & other on-chain types)
• Micheline Library (to decode and translate data found in contract calls, storage, & bigmaps)
• RPC Library (to access the Tezos Node RPC)
• Codec Library (to construct and serialize operations)
• Contract Library (for smart contract calls and tokens)
• Signer Library (to sign transactions local or remote)
• Helpers (i.e. an efficient base58 en/decoder, hash maps, etc.)

TzGo Links

• Docs: https://docs.tzpro.io/sdks/tzgo
• Repository: https://github.com/blockwatch-cc/tzgo

TzCompose: Automation Framework

TzCompose is a tool for defining and running complex transaction sequences on Tezos. It is part of the TzGo SDK. With TzCompose, you use a YAML file to configure pipelines from different kinds of tasks and then run a single command to execute these pipelines. It works on all Tezos networks and makes it easy to clone contracts and transaction sequences between them.

Developers use it for:

• smart contract deployment and maintenance
• traffic generation for protocol and application testing
• automating test-case setups
• cloning contract deployments and setup logic between networks

TzCompose Links

• Documentation: https://docs.tzpro.io/sdks/tzgo/tzcompose
• Repository: https://github.com/blockwatch-cc/tzgo/tree/master/cmd/tzcompose

TzIndex: Tezos Indexer

Blockwatch TzIndex is our battle-tested Tezos indexer. Available as open-source software since 2019, TzIndex has not only remained a constant as a reliable Tezos indexing solution, it also inherently supports decentralization as it can be run on both personal laptops and enterprise-grade servers. Furthermore, cross-compatibility between the TzIndex API and the hosted TzStats Data API contributes to a great developer experience by allowing for seamless switching between the two.

Core Features

• supports protocols up to Nairobi (v017)
• indexes and cross-checks full on-chain state
• feature-rich REST API with objects, bulk tables and time-series
• auto-detects and locks Tezos network (never mixes data from different networks)
• indexes all accounts and smart-contracts (including genesis data)
• follows chain reorgs as they are resolved
• can passively monitor for new blocks
• self-heals broken node connections (retries until node RPC comes back up)
• API supports CORS and HTTP caching
• high-performance embedded data-store
• flexible in-memory caching for fast queries
• automatic database backups/snapshots
• configurable HTTP request rate-limiter
• flexible metadata support

TzIndex Links

• Documentation: https://github.com/blockwatch-cc/tzindex/blob/master/README.md
• Repository: https://github.com/blockwatch-cc/tzindex

TzPro: Tezos Infrastructure

TzPro is a suite of powerful APIs and SDKs to access the Tezos network. With TzPro, you can easily access the entire landscape of Wallets, Tokens, DeFi and NFT protocols on Tezos. It offers the most comprehensive API solution for Tezos DeFi and NFT ecosystems as it unifies access to the broadest set of protocols and markets available today.

It offers a unified data model for common building blocks such as tokens, DEX pools, staking farms, lending protocols, NFT markets and identity solutions. Activity from different protocol implementations is indexed and mapped to this unified model so that you can seamlessly query live and historic data on all positions, trades, prices, wallet balances, transfers and more across many supported protocols.

TzPro combines 3 layers of APIs under a single roof:

• Tezos RPC (backed by load balanced Rolling and Archive mode nodes)
• Tezos Index API (our well-known legacy API for Tezos Layer 1 data)
• TzPro Token, DeFi, NFT and Identity APIs

The TzPro APIs

To best capture the broad range of protocol ecosystems on Tezos in an easy to navigate form, we split the TzPro API into sectors. As blockchain use cases and Tezos itself evolve, we're adding support for more sectors and extend existing ones:

• 🌮 Tezos Index API – transactions, wallet and contract state, bakers info, etc.
• 🦊 Tezos Token API – everything about financial assets, NFTs, LPTs, domains, minting, transfers, metadata.
• 🦄 Tezos DeFi API – in-depth activity, positions, prices across all DEX, farming and lending protocols.
• 🐳 Tezos NFT API – sales, listings, auctions, prices and analytics across all NFT markets.
• 🦓 Tezos Identity API – Domains, Profiles, smart contract metadata and curated metadata on well known wallets.
• 💰 Tezos Price API – fiat prices, token prices, NFT prices.
• ⚙️ Tezos RPC – ground-truth & mempool data from rolling or archive nodes with the ability to simulate and broadcast transactions.

Read the TzPro developer documentation on docs.tzpro.io for details on all TzPro API endpoints.

Support

We love to hear what you're building with Blockwatch. Come talk to us on our Discord dev support server, message us on X, or send a mail to hello at blockwatch.cc. For software licensing, contact us via licensing at blockwatch.cc.

We're thrilled to announce the latest addition to the Blockwatch TzGo SDK: TzCompose, an automation framework for complex transaction sequences on Tezos.

With TzCompose, you write YAML files to configure pipelines from different kinds of tasks, then run a single command that automatically executes these pipelines on-chain. TzCompose works across all Tezos networks and even supports cloning contracts and transaction sequences between them.

Use TzCompose for automating workflows such as:

• smart contract deployment and maintenance
• traffic generation for protocol and application testing
• test-case setups
• cloning contract deployments and setup logic between networks

💡 Whether you're a dapp developer, protocol engineer, generative artist or any other enthusiast building on Tezos – TzCompose will make your life so much easier.

Getting Started

TzCompose is available as part of the Blockwatch TzGo SDK  and you need a Go compiler >=1.18 to use it. The easiest way to run TzCompose is

# directly from the Git repo
go run blockwatch.cc/tzgo/cmd/tzcompose

# or by installing it
go install blockwatch.cc/tzgo/cmd/tzcompose

Then, set an environment variable containing a private key TZCOMPOSE_BASE_KEY, which will serve as your base account from which any other account will be derived and funded. See Wallets for details. TzCompose works with any RPC node (-rpc url), but if you use clone or connect the default TzPro servers, keep in mind that you need a 🔑 TzPro API key (simply export it via TZCOMPOSE_API_KEY). Don't have a key yet? No problem, you can create one for free at TzPro.

Testnets

If you're deploying on a public testnet such as Ghostnet, use Teztnets faucets provided by Oxhead Alpha to fund your base account. You can create a fresh key directly with TzGo:

go run blockwatch.cc/tzgo/examples/key gen edsk

Sandbox

If you're running tzcompose with a local sandbox this is not necessary as your sandbox environment will take care of funding your accounts. We've added a simple script for starting and stopping a local Flextesa sandbox to TzGo at ./scripts/sandbox.sh. To find an account's key on a sandbox run

docker exec tezos_sandbox flextesa key-of-name alice

Using TzCompose

TzCompose executes configurations from a single file -f file.yaml, a single directory -f ./examples/fa, or all subdirectories -f ./examples/.../ in which case all YAML files will be read in filesystem order. Specify your Tezos Node URL with the -rpc flag, e.g. https://rpc.ghost.tzpro.io or http://localhost:20000.

go run blockwatch.cc/tzgo/cmd/tzcompose [cmd] [flags]

Env
  TZCOMPOSE_BASE_KEY  private key for base account
  TZCOMPOSE_API_KEY   API key for RPC and index calls (optional)

Flags
  -f file
      configuration file or path (default "tzcompose.yaml")
  -file file
      configuration file or path (default "tzcompose.yaml")
  -resume
      continue pipeline execution
  -rpc string
      Tezos node RPC url (default "https://rpc.tzpro.io")
  -h  print help and exit
  -v  be verbose (default true)
  -vv
      debug mode
  -vvv
      trace mode

Available Commands

tzcompose offers several commands to create, test, and run pipelines.

• clone: clone transactions starting from the origination of a contract
• validate: validate compose file syntax and parameters
• simulate: simulate compose file execution against a blockchain node
• run: execute compose file(s) sending signed transactions to a blockchain node
• version: print version and exit

Available Tasks

A pipeline defines a list of tasks that are processed in order. Each task produces a transaction.

• batch: send multiple transactions as single operation
• call: send smart contract call
• delegate: delegate to baker
• deploy: create smart contract
• double_endorse: force a double endorsement slash
• register_baker: register as baker
• token_approve: approve token spender
• token_revoke: revoke token spender
• token_transfer: send token transfer(s)
• transfer: send tez transfer(s)
• undelegate: remove delegation from baker
• wait: wait for condition

Deployment Examples

You can find a list of example files under examples/tzcompose/ which give you an idea about how various deployment scenarios work. Feel free to test them on a sandbox or public testnet.

• Transfer – simply transfers tez between accounts
• Delegate – registers a baker and delegates to it
• Token – launches FA1.2 and FA2 token ledgers, mints and transfers tokens
• Tether –  deploys a copy of Tether USDt
• HicEtNunc – deploys Hic and Teia markets for NFT trading
• Quipu-v2 – deploys a Quipuswap V2 DEX for token trading

Dive Deeper

Read our documentation and start building your own scenarios. We tried making tzcompose as easy to use as possible, but it may still have a few rough edges. Please report back to us if you find yourself lost with transaction errors or if you have ideas on how to improve tzcompose further. After all the current release is an alpha and we're just getting started.

To contribute and ask questions contact us on X and Discord or send us a PR on Github. We highly value your contribution.

Stay safe out there.

All TzStats API and RPC endpoints will be officially shut down on October 31st, 2023 as we're migrating to our brand new TzPro service that brings with it several exciting new features. Migrating to TzPro is easy – learn how to do so below.

What's TzPro?

TzPro is a superset of the TzStats API. All TzStats endpoints are still available and free to use. In addition, TzPro offers a unified data model for common layer 2 building blocks such as tokens, DEX pools, staking farms, lending protocols, NFT markets and identity solutions. Activity from different protocol implementations is indexed and mapped to this unified model so that you can seamlessly query live and historic data on your DEX and NFT positions, trades, token prices, wallet balances, token transfers and more across many supported protocols.

With TzPro, we combine 3 layers of APIs under a single roof:

• Tezos RPC (backed by load balanced Rolling and Archive mode nodes)
• Tezos Index API (our well-known legacy API for Tezos Layer 1 data)
• Brand NEW TzPro Token, DeFi, NFT and Identity APIs

Coming from the TzStats API, all API routes you're familiar with still exist and the data remains the same. In addition, TzPro offers a wealth of new data on tokens, DeFi, NFT, Tezos Domains and TzProfiles. Read about the full spectrum of available services and API endpoints on the TzPro docs.

Want to read more about TzPro? Check out the announcement post, and some of our threads on X:

• New: Blockwatch TzPro
• A short thread on our game-changing TzPro API

How to Migrate?

1. Create a free TzPro Account

Sign up for your free TzPro account on the Sign Up page. Enter your email address, password, and your first and last name. Agree to the Terms and if you like, opt in to receiving the Blockwatch newsletter so we can keep you up to date.

2. Activate Account

Once you've created your new account, please activate it by clicking on the email verification link we've sent to you. Check your spam folder in case you don't see the mail. In the rare case that you still don't see it, reach out to us via hello at blockwatch dot cc and we'll get back to you as soon as possible.

3. Generate API Key

With a TzPro account, you're able to create and manage TzPro API keys. After logging in, click on API Keys on the left hand sidebar. Click on Add key on the top right of the table to create a new key. Choose a short and unique name with a maximum of 32 characters.

You can configure how traffic is handled by choosing an environment (Development, Staging, or Production) and target network (Mainnet, Staging, Ghostnet, ...). Note that currently all keys work regardless of selected network when you call api.{network}.tzpro.io.

Alright, you're set! ☕

Now, send your API key with every API request using one of two methods:

• as HTTP header X-API-Key
• as query parameter api_key=YOUR_API_KEY

If you use our TzPro-Go SDK you can configure it to use your API in two ways:

# (recommended) export your key as environment variable
export TZPRO_API_KEY=__your_key__

# (optional) or configure the TzPro client directly
tzpro.DefaultClient.WithApiKey("__your_key__")

In Javascript you can use the Request Web API

function fetchTzpro({url, key = ''}) {
  let req = new Request(url);
  req.headers.set('X-API-Key', key);
  return fetch(req);
}

TzPro API keys offer additional security features to secure your keys. Read TzPro API Access for detailed information.

Get In Touch

If you have questions, suggestions or problems using the API, you can reach us in our Discord developer support channel, DM us on X or send a mail to hello at blockwatch.cc. We're excited to see what you build on top of TzPro, so please share your projects and feel free to ask for help and advice.

TzGo is Blockwatch's low-level Tezos Golang SDK for reliable, high-performance applications with a focus on correctness, stability, and compliance with the Tezos protocol. Available on GitHub blockwatch-cc/tzgo.

New: TzGo SDK v1.17

TzGo now ships with support for the latest Tezos protocol live on mainnet – v17 Nairobi – including support for all of its data types, hashes, RPC calls, and transaction encoding. Other than with more heavy-weight protocol upgrades, Nairobi fortunately required only few changes in TzGo. This gave us the opportunity to improve two major areas:

1) Integration of a new transaction simulation RPC endpoint for more precise estimations on how much gas, storage, and fees a transaction must pay to not fail.

2) Integration of the TzGen code generator which enables creating Golang struct types and smart contract interface bindings based on a contract's entrypoint specification. The generated code can be used to deploy, bootstrap, and call any chosen smart contract and interact with all of its entrypoints including views. The bindings read/decode data found in transaction parameters, transaction receipts (storage and bigmap updates), and contract storage as well as write/encode type-conform parameters for sending transactions.

TzGen started as an external project developed by Jean Schmitt (Ubisoft). It was released as OSS under the MIT license and is using TzGo primitives. Since TzGen is less known and unmaintained, we decided to integrate TzGen into TzGo by refactoring and extending it towards common type and naming conventions to avoid code duplication. The integration will be improved further in future TzGo releases.

💡 TzGen is a game changer for Tezos Go developers as they no longer have to work with complicated Micheline types or any Tezos internals. Instead you can use native Go language features to easily interact with Tezos contracts as if they were just another network service.

Simulating Transactions

Tx simulation executes a complete transaction with a fake signature against the current state of the blockchain and returns the result. It can be used for different reasons such as predicting contract execution results, but in TzGo we use it to estimate optimal costs. Each time you send a transaction, TzGo simulates its execution to identify the actual execution costs in terms of gas usage and storage allocation. It's important to know this because a transaction has to set gas limit and storage limit. Too high limits cost more fees and when limits are too low, the transaction fails and the user wasted all fees.

For a long time, the only way to simulate was to call the RunOperation() RPC. This still works and is still used in TzGo when you request simulation on an old block, however, it cannot look into the future. Why is that important, though? To optimize gas costs, Tezos introduced a contract cache a while back, which essentially keeps the code of a few often used contracts in memory and avoids loading & deserializing the contract from disk. When a contract is not in the cache, the first transaction that loads it has to pay extra gas for deserialization.

Now, RunOperation might implicitly use the cache, but it does not know which contracts may be cached at the time when the transaction is later included on-chain. A new RPC called SimulateOperation() is specifically made for this case. By default it looks 5 blocks ahead, trying to precisely predict the future state of the contract cache based on heuristics. Starting from v1.17.2, TzGo makes use of this new RPC whenever you send a transaction under default settings. You can control how far into the future an estimation is done via the SimulationOffset parameter in CallOptions (default of 5 is fine).

// create and init a new TzGo RPC client
c, _ := rpc.NewClient("https://rpc.tzpro.io", nil)
_ = c.Init(ctx)

// create a signer from an in-memory private key sk
sign := signer.NewFromKey(sk)

// you can attach the key to your client or pass it in CallOptions like below
c.Signer = sign

// create simple tez transfer
_ = codec.NewOp().WithTransfer(recv, amount)

// Single-step operation sending (recommended by TzGo)
//
// This completes, simulates, signs, broadcasts the transaction.
// internally, the new RPC call SimulateOperation is used.
_, _ = c.Send(ctx, op, &rpc.CallOptions{
    Confirmations: 2,
    MaxFee: 10000,
    TTL: 100,
    SimulationOffset: 5, // control how many blocks in the future to simulate
    Signer: sign,
})

Note that alternatively, if you want more control over the transaction construction process, or if you'd like to simulate only, instead of calling Send() you can run:

// you probably want to have TzGo fill in missing data like counter and reveal
c.Complete(ctx, op, sk.Public())

// then simulate the call (this calls RunOperation internally)
c.Simulate(ctx, op, &rpc.CallOptions{
    Confirmations: 2,
    MaxFee: 10000,
    TTL: 100,
    SimulationBlockID: rpc.Head,
})

With the new simulation method, we were able to run complex contract calls and even large payout batches with 400 accounts as destination with the exact gas limit value that came from the simulation (no extra safety margin). In practice it is still a good idea to keep the small default margin of 100 because contract state may change between the time you simulate and the time of the execution.

Using TzGen

TzGen is a code generator for smart contract interface bindings that can take the code of your contract (or just the address if it is already deployed) to create interface methods and related types in Go. It's similar to how Protobuf or MsgPack generators work, but instead of a protocol spec it uses the contract's Micheline entrypoint type definitions. When using TzGen you never ever have to worry about details of Micheline or TzGo. TzGen produces all interfaces and types you'll need to call any of your contract's entrypoints, read its storage and bigmap entries.

So how does it work? You can run tzgen manually which will write an auto-generated Go source file that you can build together with your project and check in to your repository.

From a deployed contract
tzgen -name <name> -pkg <pkg> -address <addr> -o <file.go>

tzgen -name hdao -pkg contracts -address KT1AFA2mwNUMNd4SsujE1YYp29vd8BZejyKW -o ./examples/hdao/contract.go

From a Micheline file
tzgen -name <name> -pkg <pkg> -src <file.json> -o <file.go>

tzgen -name hello -pkg contracts -src ./hello.json -o ./examples/tzgen/hello.go

You can also use tzgen in combination with the go generate tool when you like to create fresh interface definitions at build time. To use go generate you need to do two things:

// 1. Add a go generate comment into a Go source code file
//go:generate go run -mod=mod blockwatch.cc/tzgo/cmd/tzgen -name <name> -pkg <pkg> -address <addr> -o <file.go>

// 2. Call `go generate ./path-to-your-pkg-or-cmd` to make Go call whatever script you have defined in 1.
go generate ./cmd/myprog

TzGen is currently a technology preview available in TzGo's master branch (untagged). We're collecting feedback from the community and will release it with the next protocol upgrade if everything works smoothly.

Changelog

For the full list of changes, please consult the changelog file in the TzGo repo.

unreleased

• add TzGen tech preview
• remove Mumbainet hashes, config and references

v1.17.1

• change gas simulation to /helpers/scripts/simulate_operation for better future estimates
• new CallOptions.ExtraGasMargin arg for manual override of the default (100)
• new CallOptions.SimulationOffset arg to control future block offset
• new Client.SimulateOperation() method to simulate execution at a future block

v1.17.0

• add Nairobi and Nairobinet constants
• update smart_rollup_cement parameters to Nairobi changes
• fix accounting for internal origination allocation burn
• update secp256k1 package
• add storage limit safety margin (100 byte)
• fix some Micheline translation bugs for nested list/comb-pair ambiguities
• fix decoding for some single-value entrypoints
• support TZGO_API_KEY env variable

TzGo Links

• TzGo repo (blockwatch-cc/tzgo): https://github.com/blockwatch-cc/tzgo
• TzGo developer docs: https://docs.tzpro.io/sdks/tzgo
• TzGo GoDoc documentation: https://pkg.go.dev/blockwatch.cc/tzgo
• TzStats-Go repo (blockwatch-cc/tzstats-go): https://github.com/blockwatch-cc/tzstats-go – TzGo's sister library for working with the TzStats Data API

🌶️ What's Hot? Meet TzPro.

We've recently announced the launch of TzPro: professional Tezos infrastructure for developers. With TzPro, we offer a suite of powerful APIs and SDKs to work with Tezos. Our aim is to provide you with infrastructure and tooling that enables a more efficient and joyful workflow so you can focus on building better software for our decentralized future.  

While the TzPro waitlist is still open, you can sign up for TzPro on https://tzpro.io. Learn more via the announcement post and don't hesitate to get started reading the developer documentation.

Get In Touch

Come talk to us on our Discord developer support channel, DM us on Twitter or send a mail to hello at blockwatch.cc. We're very interested in your feedback and suggestions. For software licensing details, contact us via licensing at blockwatch.cc.

At Blockwatch, we've been working closely with the Tezos protocol ever since we launched our Tezos indexer and explorer. For the past years, we've greatly improved our Tezos products and the infrastructure around them. Now, we're ready to embark on the next stage of our journey: enriching your developer experience. Welcome to TzPro – a set of powerful APIs and SDKs to work with Tezos more efficiently. With TzPro, we offer the resilience, speed, and flexibility required to deliver robust apps and services that meet professional requirements. Effortlessly navigate through Tezos' intricacies and keep focused on building the best version of your app or service.

As Tezos persists in its relentless pursuit of protocol evolution and with recent advancements in the area of scaling in particular, we anticipate growth in the deployments and user counts of professional apps on Tezos across popular sectors such as DeFi, NFT, DAOs, and more. We're excited to contribute to this new wave of activity and business on Tezos by providing you with the APIs and tools you need to go to market faster and develop a better product.

What's the timeline?

TzPro has been in production use by a few pre-launch partners for some time. Today we're opening our services to a broader group of developers and ecosystem members with an exclusive waitlist.

Secure your spot for a free TzPro account and TzPro API key by signing up on tzpro.io today.

Over the next weeks, we're onboarding small batches of new developer teams and will listen carefully to your feedback. We'll continue to grow TzPro to support more on-chain protocols and more API features in the coming months.

What's in the box?

TzPro offers everything you need to be successful on Tezos. It's by far the most comprehensive API solution for Tezos DeFi and NFT ecosystems as it unifies access to the broadest set of protocols and markets available today.

TzPro offers a unified data model for common building blocks such as tokens, DEX pools, staking farms, lending protocols, NFT markets and identity solutions. Activity from different protocol implementations is indexed and mapped to this unified model so that you can seamlessly query live and historic data on all positions, trades, prices, wallet balances, transfers and more across many supported protocols.

But wait, there is more! TzPro combines three layers of APIs under a single roof:

• Tezos RPC (backed by load balanced Rolling and Archive mode nodes)
• Tezos Index API (our well-known legacy API for Tezos Layer 1 data)
• Brand NEW TzPro Token, DeFi, NFT and Identity APIs

Parts of the TzPro APIs will be open and free for the Tezos community including access to new API features for individual wallet addresses. We want you to feel confident in migrating over from the TzStats API and have a richer experience. Only an API key is required for TzPro. Other parts of the TzPro API will be available as a paid subscription. If you run a data scraper or you want to filter and compare across DEX pools and NFT markets, that's what the paid version is for.

The TzPro APIs

To best capture the broad range of existing protocol ecosystems on Tezos in an easy to navigate form, we split the TzPro API into sectors. As blockchain use cases and Tezos itself evolve we'll add more sectors and extend existing ones:

• 🌮 Tezos Index API – transactions, wallet and contract state, bakers info, etc.
• 🦊 Tezos Token API – everything about financial assets, NFTs, LPTs, domains, minting, transfers, metadata.
• 🦄 Tezos DeFi API – in-depth activity, positions, prices across all DEX, farming and lending protocols.
• 🐳 Tezos NFT API – sales, listings, auctions, prices and analytics across all NFT markets.
• 🦓 Tezos Identity API – Domains, Profiles, smart contract metadata and curated metadata on well known wallets.
• 💰 Tezos Price API – fiat prices, token prices, NFT prices.
• ⚙️ Tezos RPC – ground-truth & mempool data from rolling or archive nodes with the ability to simulate and broadcast transactions.

Read the TzPro developer documentation on docs.tzpro.io for details on all TzPro API endpoints.

TzPro Tezos SDKs & Developer Tools

We are developers ourselves and we are passionate about performance and ease of use. Our aim is to provide you with infrastructure and tooling that enables a more efficient and joyful workflow so you can focus on building better software for our decentralized future. For this reason we offer a set of tools that help you work with Tezos on-chain data more easily.

• 🛠️ TzGo SDK: our Golang library for working directly with the Tezos Node RPC
• 🧰 TzPro SDK: our official Golang SDK to use for integrating TzPro APIs into your apps and services.
• 🔍 TzIndex: high-performance zero-config Tezos indexer available as OpenSource and PRO version.
• 🌐 TzProxy: a smart caching proxy for expensive Tezos RPC requests that can serve blocks 10-25x faster then a Tezos Node.
• 🎥 TzEvents: scalable real-time event streaming service for WebSocket and SSE.
• 🔧 TzAlias: smart contracts metadata toolbox.

What's the Cost?

The free TzPro community plan includes up to 100,000 API calls per month. Professional users and teams can subscribe to Pro and Max plans with higher limits and more features. We'll make more details available on the TzPro website soon. For now, make sure to secure your spot on the TzPro waitlist because there will be special discounts for early adopters.

Let's Talk

Talk to us on our Discord developer support channel, DM us on Twitter or send a mail to hello at blockwatch.cc. For software licensing details, please contact us via licensing at blockwatch.cc.

TzGo is Blockwatch's low-level Tezos Golang SDK for reliable, high-performance applications with a focus on correctness, stability, and compliance with the Tezos protocol. Available on GitHub blockwatch-cc/tzgo.

New: Blockwatch TzGo v1.16

With the latest TzGo version v1.16, we're providing compatibility with Tezos' 13th protocol upgrade Mumbai, incorporating new data types, hashes, RPC calls, and transaction encoding for tz4 and sr1 addresses and smart rollups. Smart rollups are supported in the RPC and codec packages along with binary encoders for all new operations introduced since Protocol Lima.

We have changed TzGo's memory layout and interfaces for address and hash types. Please note that this is a breaking change: due to the new encoding data writing using earlier versions of TzGo cannot be read with the new Binary Unmarshaler (use address.Decode(b []byte) instead). The new format allows us to save one allocation and 24 bytes per address/hash, which results in the minimal possible sizes of 21 bytes for addresses and 32 bytes for hashes. The savings are ~50% less heap memory usage and one out of two memory allocations (instead of slices, we use fixed length Go arrays now). A positive side effect is that addresses and hashes can now be used as Golang map keys since byte arrays are directly comparable by the compiler.

As a second huge improvement we have reimplemented the protocol activation database used internally to calculate start/end blocks of cycles, voting, snapshot blocks etc. After multiple protocols have changed cycle length it's no longer straightforward to do these calculations without knowing when exactly the switch happened. This change also makes TzGo more upwards compatible since the library does not have to be aware of future protocols anymore. Applications built on top just have to follow a simple protocol to update the activation database when a new protocol is detected (this can even be done at runtime). This enables simpler support for Mondaynet and Sandbox testnets and allows to run applications across mainnet upgrades without forced redeployment.

For a full changelog, please consult the changelog file on the TzGo GitHub repo.

Feature Set

TzGo contains a full set of features to read, monitor, decode, translate, analyze and debug data from the Tezos blockchain, in particular from Tezos smart contracts. It is structured into packages that provide multiple layers of composable building blocks for maximum freedom of choice and composability. Currently, TzGo contains the following 6 core packages:

• a low-level Types library to handle hashes, addresses, keys, signatures other types found on-chain
• a powerful Micheline library to decode and translate data found in smart contract calls, storage, and bigmaps
• an RPC library for accessing the Tezos Node RPC
• a Codec library to construct and serialize operations
• a Contract library for smart contract calls and tokens
• a Signer library to sign transactions local or remote
  helpers like an efficient base58 en/decoder, hash maps, etc.

Roadmap

We are committed to maintaining TzGo and keeping it up to date with new Tezos protocol upgrades, further improving its performance, and providing a best-in-class developer experience. The next version of TzGo will come with support for protocol Nairobi. Additionally, we have a few exciting features in the pipeline which will make the life of development teams building on Tezos even easier. Follow us on Twitter and join our Discord for announcements.

Links

• TzGo repository (blockwatch-cc/tzgo)
• TzGo developer documentation
• TzStats-Go repository (blockwatch-cc/tzstats-go) – TzGo's sister library for working with the TzStats Data API

Support

Developer happiness is at the core of what we do and we hope that using TzGo brings you joy. Come and talk to us via Discord and Twitter, or directly open a GitHub issue. Happy building!

We're happy to announce the latest open-source release v16 of Blockwatch's all-in-one zero-conf Tezos indexer TzIndex which is now fully compatible with the Tezos Mumbai protocol. This release includes support for all smart rollup operations and tickets along with significant performance improvements. Available on GitHub blockwatch-cc/tzindex.

Mumbai and Smart Rollup Support

With protocol Mumbai, Tezos' core engineering teams have laid the groundwork to a faster and more scalable Tezos network. In addition to 15 second block times, Mumbai introduced the next generation of highly scalable optimistic rollups called Smart Rollups that are enshrined directly into the Layer 1 protocol. This paves the way for anyone to develop and permissionlessly deploy an interoperable Layer 2 solution on top of Tezos. Rollups can be developed in various programming languages, not just OCaml like the Tezos core protocol itself. This enables the implementation of entire dapps as rollups (or app chains). This is particularly interesting for dapps that require much higher throughput than Tezos Layer 1 can sustain. All rollups can benefit from Tezos' security assurances and can share assets with Tezos L1 and other rollups based on tickets (relevant ticket bridges are under development). At the current state, rollups are not production ready because performance, tooling and standards are still missing. It's a bit early to assess when all pieces will be ready, but it's a great time to start experimenting and learning what may become possible.

Blockwatch TzIndex v16 offers initial indexing support for tz4 addresses (they use BLS keys and are supposed to be used in rollups and not L1) and sr1 addresses (smart rollups) as well as their many operations. Transaction rollups (TORU) have been deprecated, but they are still supported in TzIndex since they have been used on testnets. Like before, we wrap rollup operations into two generic transaction types rollup_origination and rollup_transaction (these do not exist in Tezos L1, just in the indexer). Rollup kind and name of the original L1 rollup operation are available as data fields along with the original data. This design is more lean and future proof than alternatives. It also means TzIndex API users can add support for Smart Rollups and upcoming zkRollups at their own convenience, while generic fields like tx fees, gas and sender/receiver are available as usual.

Besides raw operation data and the identity of rollups, TzIndex does not track rollup state (inbox, refutation games, etc.). We closely monitor the progress of tooling around rollups and may add additional features as necessary or when standards for bridging assets emerge.

Performance Improvements

We expect Tezos to grow significantly in the near future as new NFT projects, games, and rollup use cases launch. Hence we've been diligently working on improving indexing performance and overall indexer maintenance costs. We're pleased to report notable advancements in these areas, courtesy of our latest compression scheme that reduces the overall database size by 10-20%. Modifications in TzGo's internal representation of address and hash types have yielded an additional 50%+ reduction in heap memory used. The new address format now always stores 21 bytes per address instead of the earlier 21+24 bytes for an extra Go slice header (similar for hashes which are now 32 bytes instead of 56). The new formats make addresses and hashes directly comparable and on top of that save one out of two memory allocations per address/hash leading to a substantial boost in performance.

Full changelog

New Additions

• Added Mumbai operations wrapped into rollup_origination and rollup_transaction
  • smart_rollup_add_messages
  • smart_rollup_cement
  • smart_rollup_originate
  • smart_rollup_execute_outbox_message
  • smart_rollup_publish
  • smart_rollup_recover_bond
  • smart_rollup_refute
  • smart_rollup_timeout
• Added transfer_ticket op as explict operation (previously this was part of the rollup package
  and wrapped inside rollup_transaction)
• Added op model fields
  • ticket_updates on eligible transactions between tz accounts (previously ticket transfer was
    limited to contracts)
  • staker, loser, winner as participants in smart rollup refutation games
• Added block model NTickets
• Added chain model TotalTicketTransfers
• Extended list of metadata alias categories
• Detect and track protocol activations independently of hard-coded block heights

Fixes

• Crash on broken call params
• Micheline type compare for nested unions, this fixes detecting allocated bigmap types - Decoding bootstrap deposits on testnets
• Exclude unknown fields from endorsement table query
• Entrypoint detection, continue on data decode errors
• Take delegation volume from flows instead of previous balance
• Parsing block height
• Micheline translate bugs
• Some entrypoint param decoding
• Nil response crash
• Account n_tx_success column name
• Gov vote index
• Stake in post-Ithaca index 15 snapshots
• Contract table lookup, limit to successful originations
• Rollup call stats query
• Handle storage burn on transfer ticket op
• Use staking balance for gov snapshots
• Handle internal results on rollup output message execution

Improvements

• List internal ops sent by a contract
• Sanitize API requests to reject zero hashes and negative heights
• Add virtual start|end_time columns to income API
• Store height in rights and income tables
• Process internal transaction after ticket transfer
• Protect against high memory usage when loading from (zero) op hash - Prevent indexer stop on unexpected rollup data
• Protect against unexpected rollup calls
• Use spendable balance for audit
• Skip internal tx when building traffic ranks
• Add more verbose init errors
• Add auto retries to RPC client
• List factory created contracts
• CLI: change log system config names
  • log.blockchain to log.etl
  • log.database to log.db
• Upgrade to Go 1.20 and TzGo to v1.16

Licensing

Blockwatch TzIndex is available under two licenses:

1. open-source MIT license with community support/best-effort updates
2. PRO license with professional support and scheduled updates. Learn more about licensing here or e-mail us at license at blockwatch dot cc.

Support

For support, talk to us on our Discord developer support channel or DM us on Twitter. We appreciate your feedback, suggestions, and comments.

In our last post we introduced TzGo v1 for Hangzhou which was the first release that also allowed developers to send transactions. With our new release you can more easily send smart contract calls, use custom signers and work with FA token standards. Oh, and it's also compatible with Ithaca. For documentation visit the TzGo docs page and for source code head over to our Github.

What's new?

At a glance, we have

• added Ithaca support (mostly new hashes, RPC calls, and operations)
• added generic smart contract call support and support for calling off-chain views
• added support for FA1.2 and FA2  transfers, allowances and views
• added cost estimation for gas, storage, fees and flexible configuration options to manage costs (i.e. overwrite simulated values)
• improved Micheline encoding and decoding performance at least 2x by reimplementing the JSON encoder for primitives
• written more examples to show how to use contract and operation sending features

Structural changes

With the introduction of Tenderbake in Ithaca there has been a multitude of changes to how transaction data is serialized (new operations and operation tags) and it makes less sense to support older protocols at the level of encoding/signing. TzGo now produces Ithaca compatible operation encodings by default. You can find (and change) the defaults at tezos/params.go:

var (
    // DefaultParams defines the blockchain configuration for Mainnet under the latest
    // protocol. It is used to generate compliant transaction encodings. To change,
    // either overwrite this default or set custom params per operation using
    // op.WithParams().
    DefaultParams = NewParams().
        ForNetwork(Mainnet).
        ForProtocol(ProtoV012_2).
        Mixin(&Params{
            OperationTagsVersion:         2,
            MaxOperationsTTL:             120,
            HardGasLimitPerOperation:     1040000,
            HardGasLimitPerBlock:         5200000,
            OriginationSize:              257,
            CostPerByte:                  250,
            HardStorageLimitPerOperation: 60000,
            MinimalBlockDelay:            30 * time.Second,
        })
)

The RPC package still supports all previous protocols, so that reading old transaction receipts remains possible. As rights and snapshots have significantly changed in Ithaca, the RPC package does a few more calls internally to determine which protocol was active at a requested block if that's important to fetch or decode the correct data.

It turned out that the wallet package we introduced in our tech preview last time was not so useful after all. We decided to move all its logic for completing, simulating and broadcasting transactions into the existing rpc package and added a new signer package with a public interface.

type Signer interface {
	// Return a list of addresses the signer manages.
	ListAddresses(context.Context) ([]tezos.Address, error)

	// Returns the public key for a managed address. Required for reveal ops.
	GetKey(context.Context, tezos.Address) (tezos.Key, error)

	// Sign an arbitrary text message wrapped into a failing noop.
	SignMessage(context.Context, tezos.Address, string) (tezos.Signature, error)

	// Sign an operation.
	SignOperation(context.Context, tezos.Address, *codec.Op) (tezos.Signature, error)

	// Sign a block header.
	SignBlock(context.Context, tezos.Address, *codec.BlockHeader) (tezos.Signature, error)
}

The signer is used by the rpc.Send() and contract.Call() family of functions, but you can always use it directly and attach the signature to an operation with op.WithSignature()

As initial signers we support an in-memory signer that can be created from a private key and a client for the Tezos remote signer interface. Feel free to add your own custom signer implementation, but it should be compatible with the interface above to use the convenience wrappers.

Sending Transactions

We made it easy to send simple transactions and even batches, but at the same time allow full control of all the details. The shortest complete example for transferring tez (ignoring errors) is this:

import (
    "context"
    "blockwatch.cc/tzgo/codec"
    "blockwatch.cc/tzgo/rpc"
    "blockwatch.cc/tzgo/signer"
    "blockwatch.cc/tzgo/tezos"
)

// load key and receiver
sk := tezos.MustParsePrivateKey("...private_key...")
to := tezos.MustParseAddress("...receiver...")

// create an RPC client
c, _ := rpc.NewClient("https://rpc.tzstats.com", nil)

// use private key to sign
c.Signer = signer.NewFromKey(sk)

// construct a transfer operation
op := codec.NewOp().WithTransfer(to, 1_000_000)

// send (will complete reveal, simulate cost, add cost, sign, broadcast, wait)
rcpt, _ := c.Send(ctx, op, nil)

// do smth with receipt, e.g. check for success, read costs
_ = rcpt.IsSuccess()
_ = rcpt.TotalCosts()

For more control, use optional features from the Op struct or provide extra call options as a third parameter to rpc.Client.Send():

// Use different protocol parameters
func (o *Op) WithParams(p *tezos.Params) *Op

// Set a custom TTL or directly set the branch to use
func (o *Op) WithTTL(n int64) *Op
func (o *Op) WithBranch(hash tezos.BlockHash) *Op

// Set custom gas, storage, fees for each operation in a batch
func (o *Op) WithLimits(limits []tezos.Limits, margin int64) *Op

// Use custom call options for rpc.Client.Send()
type CallOptions struct {
    // number of confirmations to wait after broadcast
    Confirmations int64         
    // max acceptable fee, optional (default = 0)
	MaxFee        int64         
    // max lifetime for operations in blocks
	TTL           int64         
    // ignore simulated limits and use user-defined limits from op.WithLimits()
	IgnoreLimits  bool          
    // custom signer to use for signing the transaction
	Signer        signer.Signer 
    // custom address to sign for (use when signer manages multiple addresses)
	Sender        tezos.Address 
    // custom block observer for waiting on confirmations
	Observer      *Observer     
}

TzGo simulates every operation before signing and then uses the simulated gas and storage costs to avoid rejection. In some cases, usually for high throughput dapps that send many transactions per block, it is possible that the Tezos node underestimates storage costs. This is a known and unresolved issue. If this happens frequently, please set custom fees/costs for such operations and disable the use of simulated values with CallOptions.IgnoreLimits = true (make sure you set a reasonable baker fee as well).

If no user-defined fees are set (default), TzGo calculates minimum fee using the minimal fee algorithm published here. Setting IgnoreLimits = true disables this behavior.

Calling Smart Contracts

TzGo v1.12 adds another package contract that simplifies working with smart contracts and FA tokens. It also allows you to batch several contract calls into the same transaction. A simple FA2 transfer can be implemented as

import (
    "context"
    "blockwatch.cc/tzgo/codec"
    "blockwatch.cc/tzgo/rpc"
    "blockwatch.cc/tzgo/signer"
    "blockwatch.cc/tzgo/tezos"
)

// create an RPC client
c, _ := rpc.NewClient("https://rpc.tzstats.com", nil)

// use private key to sign
c.Signer = signer.NewFromKey(tezos.MustParsePrivateKey("SK..."))

// constuct a new contract
con := contract.NewContract(tezos.MustParseAddress("KT1..."), c)

// construct an FA2 token (use token id 0)
token := con.AsFA2(0)

// construct simple transfer arguments
args := token.Transfer(
    tezos.MustParseAddress("tz..."), // from
    tezos.MustParseAddress("tz..."), // to
    tezos.NewZ(1_000_000),           // amount
)

// execute the call (will complete reveal, simulate cost, add cost, sign, broadcast, wait)
rcpt, _ := con.Call(ctx, args, nil)

In more complex scenarios you can add multiple transfers to the same FA2 transfer list

// construct complex transfer args
args := contract.NewFA2TransferArgs()
args.WithTransfer(from_1, to_2, token_id_1, amount_1)
args.WithTransfer(from_2, to_2, token_id_2, amount_2)

// optional, optimize args (for minimal parameter size)
args.Optimize()

// execute the call
rcpt, _ := con.Call(ctx, args, nil)

or you can batch multiple calls

// append multiple calls (silly example, but you get the point)
args := []contract.CallArguments{
    fa2token.AddOperator(owner, operator),
    fa2token.Transfer(from, to, amount),
    fa2token.Transfer(from, to, amount),
    fa2token.RemoveOperator(owner, operator),
}

// execute all calls in a single batch transaction
rcpt, _ := con.CallMulti(ctx, args, nil)

Outlook

With the current release, TzGo has all the tools you need to write complex Tezos applications in Go. A few things are left on our TODO list which will make TzGo even more useful in specific scenarios. Upcoming releases may include

• a more powerful Observer that can filter by addresses and monitor the Tezos mempool (all the basics already exist in packages tezos and rpc)
• create keys and signers from mnemonic seed phrases and faucets
• helpers to access contract storage and bigmaps more easily
• reading and writing contract/token metadata

Conclusion

If you like TzGo or if you have an idea on how we can improve or extend it in the future, please reach out on Twitter, Discord or email.

Happy building!

Recently we published the first demo of our Tezos Polywrapper which allows easy integration of Tezos into any application written in any language. Today, we are happy to provide an update on our progress and get you started on how to use Tezos Polywrappers in your project or create wrappers for your own Tezos dapps.

First, a recap.

What's Polywrap?

• Polywrap is a developer toolchain that enables easy integration of web3 protocols into any kind of application, eliminating the need for client-side SDKs
• By using dynamically loadable WebAssembly modules with a standardized and discoverable GraphQL interface, Polywrap makes dapps more lightweight and multi-platform

Why Tezos Polywrapper?

• Tezos Polywrappers offer a plug-and-play solution that enable straightforward Tezos integration
• By lowering the barrier to interacting with the Tezos blockchain and Tezos dapps, developer adoption is increased

Our aim with the Tezos Polywrap integration is to make it easy for developers outside of the Tezos ecosystem to work with Tezos and easy for developers already onboarded to Tezos to create wrappers for Tezos dapps. Ultimately, we provide examples of wrappers for popular Tezos dapps to inspire you to build wrappers for your own or favorite Tezos projects.

Prototype Update

Previously, we hosted the first version of our core Tezos Polywrap plugin prototype on a private repository to which we added external contributors and reviewers on demand. Now that the project is maturing, we are making it available on a public GitHub repository.

• Main repo: https://github.com/blockwatch-cc/polywrap-tezos
• Documentation: https://blockwatch.gitbook.io/polywrap-tezos

The polywrap-tezos repo is a direct fork of Polywrap's upstream repo for integrations to which we will send pull requests once our implementation becomes stable.

Since the initial announcement, we have extended our core Tezos Polywrap plugin with external wallet support. Behind the scenes, we use Temple Wallet through its Taquito implementation and export the following functions for use by wrappers (we plan on using Beacon SDK later).

Support for read-only queries to the Tezos blockchain:

- getPublicKey
- getPublicKeyHash
- getRevealEstimate
- getTransferEstimate
- getOriginateEstimate
- checkAddress
- getBalance
- getContractStorage

Support for interacting with the Tezos blockchain:

- callContractMethod
- callContractMethodAndConfirm
- signMessage
- transfer
- transferAndConfirm
- originate
- originateAndConfirm

In preparation for dapp support, we've built wrappers for two more Tezos dapps in addition to the Tezos Domains plugin. There are now:

• Tezos Domains – as Polywrap Plugin written in TypeScript
• Harbinger Oracles – as Polywrap Wrapper written in AssemblyScript
• Hic et Nunc – in progress, as Polywrap Wrapper written in AssemblyScript

As part of our final milestone, we will extend this with a wrapper for the upcoming V2 of the Quipuswap DEX.

Using Wrappers

To use any Polywrapper in your application, you need to install the Polywrap JavaScript client. Note that Polywrap's goal is to support every major programming language, with upcoming support for Rust. Blockwatch is exploring developing a Go toolchain for Polywrap.

Install via npm:

npm install --save @web3api/client-js

The Polywrap JS client works both in Node.js and browser apps. Currently, Chrome and Brave are the only supported browsers. This will be amended soon.

Initialize the client:

import { Web3ApiClient } from '@web3api/client-js';
const client = new Web3ApiClient();

Great, you can now interact with any Polywrapper from any application such as our example wrappers found in the polywrap-tezos repo's tezos directory.

Creating Wrappers

To start developing your own Tezos Polywrapper, clone our fork of the Polywrap integration repository.

git clone https://github.com/blockwatch-cc/polywrap-tezos

Note that as long as Polywrap Tezos is in alpha stage, we are hosting our code's most recent version on our own repo. Once a stable release is ready, we will make the official version available directly on the upstream Polywrap repo.

Check back for our upcoming guide on creating a Tezos Polywrapper or dive right into the code of the example wrappers.

Support

If you're interested in using or creating Tezos Polywrappers and have questions or suggestions, feel free to reach out to us via Discord or Twitter. For questions directly related to the Polywrap platform, join the Polywrap Discord.

In May '21, we introduced the first public release of TzGo – our low-level Tezos Golang SDK with a focus on correctness, stability, and compliance with the Tezos protocol.

TzGo's initial release included libraries for Tezos types, Micheline decoding, and RPC access. While this read-only functionality sets a crucial foundation to access Tezos on-chain and node-level data, we have been working on upgrading TzGo to support transaction creation, signing, simulation, and injection.

We’re excited to announce the first release candidate for TzGo Version 1.0, the first version of TzGo that allows you to prepare and send transactions. All types and interfaces are engineered to be easily composable. We start with essential low level types/functions for public and private key handling, operation encoding/signing/broadcasting, and mempool/block monitoring.

This release candidate is already fully functional and ready to be used for sending transactions. Low-level interfaces are stable, however, right now you have to roll your own cost estimations and account status handling, but we're adding this on the way to the final release. We're also going to add support for wallet interactions and a higher-level contract call interface.

Structure

TzGo is structured into packages that provide multiple layers of composable building blocks for maximum freedom of choice and composability. Application programmers can directly use low-level types for greater control of details and communication patterns or use more convenient high-level functionality which hides details and may send a number of RPC calls in the background.  Currently, TzGo contains the following 5 core packages:

• tzgo/tezos low-level types for hashes, addresses, keys, signatures other types found on-chain
• tzgo/micheline to decode and translate data found in smart contract calls, storage, and bigmaps
• tzgo/rpc for accessing the Tezos Node RPC
• tzgo/codec to construct and serialize all operation types
• tzgo/wallet for account and operation management
• helpers like an efficient base58 en/decoder, hash maps, etc.

To simplify complex use cases, we will introduce a lot more higher-order functionality in the next release candidate.

Getting Started

TzGo is free and easy to use. It is open-source software and available on Github, ready for you to get started right away:

• Repo: https://github.com/blockwatch-cc/tzgo
• TzGo Developer docs
• TzGo GoDoc documentation

Download and install TzGo, import as:

go get -u blockwatch.cc/tzgo

import (
	"blockwatch.cc/tzgo/codec"	
	"blockwatch.cc/tzgo/tezos"
	"blockwatch.cc/tzgo/micheline"
	"blockwatch.cc/tzgo/rpc"
	"blockwatch.cc/tzgo/wallet"	
)

Roadmap

In RC-0 we have focused on low-level support for handling private/public keys, key generation/encryption, transaction creation/sending/validation/broadcasting and mempool monitoring. Throughout Q1 2022 we will enable remote signer and smart contract support and make TzGo's new features even more easier to use. Please consult our official TzGo developer docs roadmap for details. We're also going to extend our developer documentation and post a series of articles with hands-on guides on how to use TzGo.

Just like for our Tezos API, TzIndex indexer, and TzStats explorer, at Blockwatch we are committed to keeping TzGo up to date with the latest Tezos protocol so that you can focus on building your products instead of worrying about protocol upgrades. Protocol Ithaca research is well on its way and we'll be adding Ithaca support to TzGo in a short while.

Developer happiness is at the core of what we do and we hope that using TzGo brings you joy. Be invited to get in touch via Twitter, Discord, or ping us elsewhere. Tell us how you use our software. Share feedback, report bugs, ask for new features.

Have a great day and happy building!

Polywrap is a Web Assembly (WASM) standard for integrating web3 protocols into any kind of application. The way Web3 works today is that developers import custom SDKs from DeFi, NFT, or DAO protocols directly into their apps and ship these SDKs on deployment. This approach, inherited from web2 devops workflows, creates several drawbacks for a decentralized trust-less system: first, traditional SDKs are limited to a specific programming language, so that devs are locked to the SDK's choice. Second, the web2 model of importing dependencies at the library/source-code level has proven to be too insecure and bulky.

Polywrap at a glance

Polywrap improves the orchestration of web3 dependencies by making web3 protocols dynamically loadable WASM modules with a standardized and discoverable GraphQL interface. Modules, also called Polywrappers, are loaded into web3 apps via a chain of trust starting at an on-chain name resolution system like ENS or Tezos Domains where a module's name and content hash are registered. After lookup, the compiled Polywrapper is loaded from a trustless content delivery system like IPFS. On-chain name registry and content hash are trust anchors that improve app security while WASM serves as a common abstraction layer to separate development language from host language. Polywrappers can be written in any language that compiles to WASM (AssemblyScript and Typescript are supported, Rust and Go are in preparation). A host app interacts with a Polywrapper by executing GraphQL requests to either query or mutate the state of a web3 protocol. It's important to know that as of now Polywrappers are entirely state-less: all state is stored on-chain and the Polywrapper mediates read and write operations.

Why Tezos Polywrapper?

At Blockwatch, we are working to increase Tezos developer adoption by enabling applications to interact more easily with the Tezos blockchain. Right now, developer adoption in Tezos is challenged by the protocol’s complexities and its frequent protocol upgrades which results in a higher barrier to entry and additional maintenance. Many developers and businesses ask us for a plug-and-play solution and we believe Polywrap can become one of the choices. It will make Tezos more accessible to a larger developer audience.

Our Tezos wrapper for Polywrap is designed to enable straightforward integration of Tezos into web3 applications. It is built in TypeScript, one of the native development languages for Polywrap modules and plugins. We aim to keep all components of our wrapper up-to-date with new Tezos protocols and at the same time keep the high-level APIs stable so that application developers don’t have to worry about low-level changes to the Tezos protocol or its RPCs.

Project Overview

Polywrap supports WASM modules and native Typescript plugins to extend its functionality to multiple blockchains. Plugins are blockchain clients that provide core functionality to interact with a chain such as reading data and sending transactions via its native RPC interface. Plugins are implemented in JavaScript/TypeScript and loaded into a Web3 host application directly. Modules are wrappers for Web3 protocols that are deployed as WASM binaries which can be loaded on-demand by an application. Each module wraps a specific contract (or family of contracts) such as a DeFi DEX or an NFT marketplace. Modules are versioned and may be deployed on IPFS. Their IPFS hash is registered as metadata on a Tezos or ENS domain such as e.g. w3://ens/v2.uniswap.web3api.eth. Wrapper describe their interface with a GraphQL schema definition that provides types and parameters for contract calls to a Web3 protocol. All business logic to execute calls (e.g. buy a Tezos domain or swap on Quipswap) is implemented within the WASM module. Host apps can either query the protocol's state (which may result in reading on-chain storage or calling on-chain or off-chain views) or mutate a protocol's state by sending contract calls. From the side of a host app this looks like a regular (async) function call. Polywrap and its WASM modules handle all the protocol-specific logic including the construction of transactions and their signing process behind the scenes.

Initially, Polywrap was designed to work with Ethereum. Hence its current module loading system is very much centered around Ethereum domains and clients. However, Polywrap’s vision has always been to become a multi-chain platform where developers can discover, deploy, and interact with various Polywrappers for different web3 protocols. We are currently extending Polywrap to support tools from the Tezos ecosystem such as Taquito and Tezos domains which will be the core building blocks of our integration going forward. Taquito will be used for interacting with the Tezos node RPC to craft transactions and read smart contract state while Tezos Domains are used to register and find Tezos Polywrap modules.

Project Status and Outlook

As a starting point we have implemented a Tezos client as a Typescript Polywrap plugin which supports connecting to the Tezos RPC to read account information, query contract data, call views, and send transactions. This plugin provides core functionality to execute query and mutation functions from Tezos Polywrap modules as well as to find Tezos modules via Tezos Domain names and load them from IPFS. Right now, our client uses a simple in-memory wallet and signer implementation which we're going to extend soon.

Additionally, we implemented a first prototype plugin for Tezos Domains which can be used to query domain data and buy new domains. For now, this is a proof-of-concept that helps us learn how to best integrate Tezos call conventions with Polywrap. Ideally, the Tezos Domains protocol will be implemented as Polywrap WASM module so it can be loaded on-demand.

Demo Application

We have prepared a small demo application that shows the functionality of our Tezos domains Polywrap integration. Currently, you can search for registered Tezos domain names across Tezos testnets and mainnet. The network to search on is detected from the TLD domain you add (.tez for mainnet and .gra or .han for one of the testnets).

• Visit the demo app here
• Github repository for the app is here

To get an idea of the developer experience, we're showcasing how some common tasks will most likely look like (note that since this is a work in progress, expect interfaces and naming conventions to change):

import { Web3ApiClient } from "@web3api/client-js"
import { tezosDomainsPlugin } from "@web3api/tezos-domains-plugin-js"

export const client = new Web3ApiClient({
  plugins: [{
    uri: "w3://ens/tezos-domains.web3api.eth",
    plugin: tezosDomainsPlugin({
      connections: {
  		granadanet: {
		  provider: "https://rpc.granada.tzstats.com",
	      supportedTLDs: ['gra']
        },
        hangzhounet: {
          provider: "https://rpc.hangzhou.tzstats.com",
          supportedTLDs: ['han']
        },
        mainnet: {
          provider: "https://rpc.tzstats.com",
          supportedTLDs: ['tez']
        }
      },
      defaultNetwork: "mainnet"
    })
  }]
})

const uri = 'w3://ens/tezos-domains.web3api.eth'

export const resolveDomainRecords = async (connection, domain) => {
    return client.query({
        uri,
        query: `
            query {
                resolveDomainRecords(connection: $connection, domain: "${domain}" )
            }
        `,
        variables: {
            connection
        }
    })
}

Outlook

In the next iteration of our research we will add support for the Beacon SDK and external Tezos wallets into the Tezos plugin. We will also create a first WASM Polywrapper module for Tezos Domains. Once we know how modules work we will start building additional Tezos Polywrappers for popular contracts such as Quipuswap, Hic et nunc, and Harbinger Oracles. Along the way we’re going to release more documentation so that developers can build and publish their own Tezos Polywrappers and make more of the great protocols in the Tezos ecosystem universally available.

If you found this article interesting and would like to stay up to date on our progress, follow us on Twitter and join our Discord. When you have an idea or opinion on how a cross-chain developer & user experience between Tezos and other chains like Ethereum should look like, please share your thoughts with us. We also have open positions for frontend and backend developers with a solid experience in working with different blockchain protocols, databases and distributed systems. Drop us a line at jobs@blockwatch.cc

Tezos is a self-amending blockchain technology that lets anybody prepare and publish a proposal for a future network protocol. Once it passes a five-stage governance process, the new protocol will automatically go live across the network.

As more services, applications and users rely on data correctness and high availability of our APIs and products, we have been investing our time in better infrastructure testing, and in particular, testing the impact of a real mainnet upgrade. We like to share our experiences and a guide for performing such tests yourself.

This guide is meant for Tezos bakers and people who are generally interested in the Tezos protocol. In a nutshell, we will cover the exact steps that let you:

• download, patch, compile and run a special test version of the Tezos node called Yes node from source
• import a snapshot to create a so-called full node
• run the protocol migration by manually baking new blocks

We walk you through this process step by step and provide command line examples so you can follow along. To get started, you need a laptop or server with about 20 GB free disk space, as well as a good internet connection since you need to download a 4 GB snapshot file plus all sources and dependencies.

Why Protocol Migration Testing

When migrating to a new Tezos protocol, a new context is produced by applying migration operations to the previous context (context = file system of Tezos blockchain state). This process is sometimes simple and fast as in Florence and Edo, but it can also be data and compute intense as in the upcoming Hangzhou migration (see Marigold's blog post for details). Hangzhou needs to rewrite large data structures which can take anywhere from 10 minutes to 3 hours.

Testnets are one piece of our testing regime where we can check if our software works as expected with new RPC endpoints and protocol features in an isolated environment. For this reason we run our own testnet nodes including testnet bakers from day one. Tezos core developers make sure that Tezos testnets perform a protocol migration early on which is very helpful because we can run our tests on small datasets first.

Recent experiences have taught us, however, that this is not enough. We also need to test migration on "real" mainnet context, i.e. the entire state imported from Mainnet, because its larger size can lead to unexpected performance and timing issues that won't be visible on a testnet. We're interested to see how the size and complexity of a mainnet context affects a smooth migration and our ability to pull data from the RPC during migration and afterwards.

So how do you simulate such kind of migration on real mainnet data?

Preparing a Yes Node for testing

As part of this process we will download a snapshot from a trusted source like https://mainnet.xtz-shots.io. Visit the page and copy the link to the Rolling Snapshot. We will need it in step 6, and the block number in step 3.

When writing this guide, the most recent available snapshot was from block 1,895,269. For you this will most certainly be different, so you need to adjust steps 3 & 6 accordingly.

https://mainnet.xtz-shots.io/tezos-mainnet-1895269.rolling

We run our examples inside Docker using Alpine Linux and an external volume to keep our state across container restarts. Besides installing some utilities and compilers the instructions should work in a similar way across other Linux distros and even MacOS homebrew.

0. Create a Docker volume, start a vanilla Alpine container, and install packages

# create a new volume to hold node and wallet data
docker volume create tezos_migration

# start an Alpine container and attach our volume to /data
docker run --name yes-node -it --rm -v tezos_migration:/data alpine sh

# inside the container, update package descriptions
apk update && apk upgrade

# and install a few packages we need to build Tezos
apk add patch unzip make gcc m4 git g++ aspcud bubblewrap \
  curl bzip2 rsync libev-dev gmp-dev pkgconf perl bash \
  hidapi-dev binutils-dev ocaml opam cargo libffi-dev \
  zlib-dev ncurses-dev openssl-dev jq

1. Check out the Tezos v11 branch and prepare all dependencies

git clone --depth 1 -b v11.0 https://gitlab.com/tezos/tezos
cd tezos
opam init --bare --disable-sandboxing --no-setup --yes
make build-deps
eval $(opam env)

2. Next, apply the yes-node patch and create a yes-wallet (the yes node is a special version of Tezos that accepts any private key for signing baked blocks; further down we still need to bake as the baker who owns the rights for a given block)

patch -p1 < scripts/yes-node.patch
dune exec scripts/yes-wallet/yes_wallet.exe -- create minimal \
  in /data/yes-wallet

3. Run the user_activated_upgrade.sh shell script to trigger the upgrade of the protocol 2 blocks after the snapshot level. In our case, e.g. 1895269 + 2 = 1895271.

./scripts/user_activated_upgrade.sh src/proto_011_PtHangz2 1895271

4. Ready to compile our patched Tezos node and CLI tools

make

5. Once that's done, create a temporary data dir for your yes-node

mkdir /data/yes-node

6. Now is the time to import the snapshot from above (this will take 25+ minutes, so feel free to fetch a coffee now)

# download
wget -P /data https://mainnet.xtz-shots.io/tezos-mainnet-1895269.rolling 

# and import the snapshot (we skip validation checking to save time)
./tezos-node snapshot import \
  --data-dir /data/yes-node/ \
  --no-check \
  /data/tezos-mainnet-1895269.rolling 

6.1 (optional) For your convenience and if you have disk space, make a backup copy of the folder. This way, you can easily perform the process multiple times.

cp -r /data/yes-node /data/yes-node-backup

7. Start your patched Tezos node without any connection.

./tezos-node run --connections 0 \
  --data-dir /data/yes-node \
  --rpc-addr localhost 

Now let's bake some blocks!

Running the Manual Migration Test

This part of the guide requires a second terminal. In Docker we can simply exec into  the running container

docker exec -it yes-node sh

1. We need to bake 2 blocks for the migration to start. Do do that, we can use the pre-defined baker aliases foundation1 – foundation8. If one of them does not work, i.e. you get an error telling you there are no baking rights, pick another baker. Lets bake a test block:

# first change to the tezos build directory
cd tezos

# then bake a test block to check everything works
./tezos-client -d /data/yes-wallet bake for foundation1 --minimal-timestamp

2. When you bake the second time the magic is set to happen. The client will only return after the block has been processed which includes the entire protocol migration. Don't abort the command. Optionally add time in front to measure runtime.

Note: Timing may be a bit tricky here. Sometimes you have to wait a bit until the second block is accepted. If you see a "timestamp in the future" error, delete the file /data/yes-wallet/blocks before you try again!

time ./tezos-client -d /data/yes-wallet bake for foundation1 --minimal-timestamp

Now take a walk or take another sip of coffee.

Meanwhile your node logs in terminal 1 should show the following output

node.protocol: 011-PtHangz2: flattening the context storage: this operation may take several minutes

and after the process is complete

node.protocol: 011-PtHangz2: context storage flattening completed

(...)

node.store: the protocol table was updated: protocol PtHangz2aRng (level 11) was
node.store:   activated on block BL...
node.store:   (level 1895271)

Congratulations! You have successfully performed a manual Tezos migration test.

Our test results

We performed the above tests on mainnet contexts at blocks 1,884,950 and 1,895,271 with both a full archive node and a rolling node from a fresh snapshot like described here.

We used a 12-core 64G bare metal server with 2x NVMe drives in Raid-1. Our Tezos node ran inside a Docker container with Alpine Linux 3.14 without CPU or mem limits and data volumes mounted via LVM. The process was mostly I/O bound for archive (80% iotop usage) and CPU bound for full node (single-core only).

Our results are

11 minutes is not bad. As Marigold said in their post, we suggest bakers run a rolling node, preferably built from a snapshot at the day of the Hangzhou protocol upgrade. That way we can keep the network downtime to a minimum.

Happy baking!

Listen to the full podcast here, and read the full transcript here.

So today we have with us, Kevin Ngo, and Jordan Ellis from Polywrap welcome to the podcast.

Ellis said, "The initial intention behind the project is to facilitate better integrations for web3 projects for applications. And the whole need for this comes from our experience building in the web3 space. We found that protocols are continuing to do a lot of the same legwork over and over again so our particular problem that we're solving is the legwork being done around developing SDKs for web3 protocols. And for example, for all your favorite let's say Ethereum smart contract protocols there are JavaScript SDKs, and we find that with this there's lots of problems that exist.

And so we can talk a bit more about that, but our main driving mission is to create this next generation SDK development experience so that you can not only support applications outside the browser and, also including the browser of course, but allow you to enable lots of other cool functionality that is not possible today with JavaScript SDKs.

And so this applies itself very well to this problem that we're discussing because in the web3 space there's a lot of friction in between smart contracts and the applications that integrate them. So that's like the efficient integration plans. That I was referring to. And then additionally we currently only see web3, really being used in-browser based applications. And so it's not very multi-platform today. And so that's the the build once and use everywhere problem that I was discussing previously."

This is a full transcript. Listen to the podcast here.

Nathan: So today I have with us, Kevin Ngo, and Jordan Ellis from Polywrap welcome to the podcast.

Jordan: Thanks for having us.

Kevin: Yeah. Thank you, thank you.

Nathan: Jordan is one of the original contributors to Polywrap and Kevin Ngo is the developer relations engineer at Polywrap. Can you gentlemen, start off and give us a little bit of the vision behind the company and how the project was conceived and give us a little overview of who you all are?

Jordan: Yeah, sure thing. The initial intention behind the project is to facilitate better integrations for web3 projects for applications. And the whole need for this comes from our experience building in the web3 space. We found that protocols are continuing to do a lot of the same legwork over and over again so our particular problem that we're solving is the legwork being done around developing SDKs for web3 protocols. And for example, for all your favorite let's say Ethereum smart contract protocols there are JavaScript SDKs, and we find that with this there's lots of problems that exist.

And so we can talk a bit more about that, but our main driving mission is to create this next generation SDK development experience so that you can not only support applications outside the browser and, also including the browser of course, but allow you to enable lots of other cool functionality that is not possible today with JavaScript SDKs.

Nathan: Now I've been in the space for a little while, as I'm sure you guys have, and there's been a lot of interoperability working groups and development shops. Can you go into the specifics of what makes your background critical to the development of this industry?

Jordan: Yeah, certainly. So speaking just for myself I had been thinking about this problem ever since I joined the web3 space. Where I come from is from a game development and also operating systems background. And I was particularly interested in really efficient toolchains for creators and for the game industry, and then also for multi-platform support so that you can build once and use it everywhere.

And so this applies itself very well to this problem that we're discussing because in the web3 space there's a lot of friction in between smart contracts and the applications that integrate them. So that's like the efficient integration plans. That I was referring to. And then additionally we currently only see web3, really being used in-browser based applications. And so it's not very multi-platform today. And so that's the the build once and use everywhere problem that I was discussing previously.

And so for myself I saw this very much so when I first joined the space and I was thinking of ways to solve it. I wanted to gain more experience.

So myself and other colleagues at the dOrg development shop over the past years we've been working with lots of web3 startups, helping them build different solutions that that they were requiring, and through doing so we realized that, yes, this is a major problem that's hindering these startups, and then additionally we found a way to solve it with the solution that Polywrap presents today. And soon after finding that magic bullet we built it at the hackathon a year ago, and then we spun up the DAO, I think seven months ago. And yeah that's where we are today.

Kevin: Thinking a little bit my background I was working on developer tools for React, then earlier this year, I became very excited about the crypto industry. I learned about polywrap through a job posting and just spoke with the team and became really excited from their background at dOrg, working on some of the major projects in web3 today. And so I chose to be a dev relations engineer, I always liked a role that's half technical and, but also half dealing with people. And in this role I'm working with a lot of our launch partners like Gnosis and WalletConnect to see how they could integrate Polywrapping to there.

Nathan: Oh, wonderful. Thanks for that clarification. Do you want to talk about the DAO? Can you explain to the listeners how you formed it, and what jurisdiction, and I think a lot of people are trying to figure out how to replicate or even just build a DAO. I'd love to hear about that.

Jordan: Yeah, definitely. We had a lot of experience starting up different DAOs in the past for client projects and we formulated an approach for this project that we thought suited it well.

First off it's not itself like an organization that has a jurisdiction. So it doesn't have a legal entity in any state. And so instead it is a combination of many different entities that are coming together in this kind of a neutral organization, which we call the Polywrap DAO and we purely just have a governance token currently.

And the aim of this Polywrap DAO is to steward the development of the toolchain and the eclipse. And so we, spun it up on Aragon. We did an initial fundraiser, which you can read about on our blog. And that's been funding all of the developers working on the project over this past eight months now, like seven to eight months.

And we have members of the DAO that are, not only the funders, but also the early contributors we call those kind of like the pre-seed builders. And then we also, as we build this project over the course of 2021 which is like what our current roadmap is set up for. And we're currently building a future roadmap So over the course of 2021, we're issuing out our what we call our dev mining allocation.

So it's basically a set of governance tokens that we're issuing to builders, along with the stable coins, they earn to to basically seed out the governance to the people, actually building the project along with the initial team of let's say like 10 people. And so it's like a gradual decentralization approach where we started off with the founding team and set up a set of funders.

And then we're expanding that set of governance holders to all the builders working on the project. And we want to continue with this approach going forward because the goal of this is to be an organization that governs a toolchain that is open, free software. That lots of people are, we hope will be dependent upon.

And we really want this to be user owned infrastructure, as much as possible. And so that's where the gradual increasing the set of token holders come from.

Nathan: So w when you survey the landscape of foundations, the ETH Foundation, the web3 foundation the Algorand Foundation, why did you guys choose to issue tokens in the States without a foundation or without any legal entity.

Jordan: Yeah, so we, we're not just doing this in the United States, it's all over the world. And yeah and the reason is because we view this more as a cooperative where these different interested parties can come together and co govern what, what happens and in the future, we may choose to spin up a Foundation, let's say for some legal reasons, but based on the legal counsel that we had this was the lightest weight and best approach to go do.

Nathan: Do you want to give a plug to the counsel who was the counsel that you guys use?

Jordan: Yeah yeah. We worked with a lawyer who was previously, or I actually, I think currently works at the Tezos Foundation. And so he was very familiar with crypto law and how the foundation model worked. And yeah we chose to go with this route.

Kevin: Yeah. And just adding on a little bit to what Jordan's saying, like having a DAO is very low. friction for us. So we can onboard very high, talented contributed very quickly. We have dev shops that have started basically the next day. And they're, motivated because they're working flexibly as contractors, but they also get a stake in the DAO itself. And yeah. I think that's a good reason for why we started as a DAO.

Nathan: So the only legal entity is really the sort of GitHub link that people use to make posts and make pull requests?

Jordan: Yeah, so it's the DAO is a collection of other organizations. And so each party that is involved in the project has their own legal personhood.

Nathan: So then the DAO is just a bunch of personal LLCs or people actually just accepting that risk themselves rather than an entity.

Jordan: Yeah, it's a personal LLCs and then also also organizations like for example, we have a few different development shops that are, getting paid through the DAO, dOrg is one of them. There's also a development shop called Node Factory. And then another one called Consider It Done. And then in addition to that the funders all have their own organizations as well. And yeah, it's a collection of different organizations just coming together to basically co-fund something and and co-govern it in the future. It's strictly just to see this project take off because all the parties are interested in the advancements that it brings to the space.

Nathan: So walk us through that a little bit. Let's say the DAO evaporates, like how does the toolchain stick around and how do people keep using the toolchain if the DAO somehow evaporates?

Jordan: Just as all other software lives on the internet our code is all open source and if something, let's say were to happen to the DAO, let's say our plan with, furthering the decentralization of governance leads it down, like a path that we didn't intend, and since all this, source code is openly available, if there was a body of users that wanted to use it, they could, fork it create their own their own organizations to steward it. Yeah there's really nothing that links the success of the toolchain to the success of the DAO's governance.

But merely I think that the the DAO was just a coordination mechanism to try and help the stewarding of this of the toolchain.

Nathan: It's really interesting. I know we'll get into the tech stack, but I'm fascinated with this as well. Is there another example, like an analogous example?

Jordan: No, I don't think so. But I think that if you think of like early developer communities that have spun up over the years, so for example, let's say Linux or yeah, I think that Linux is a great example because It was started very grassroots. There, there were a few initial contributors. They would collaborate over Git but the governance of the project was all pretty centralized. Cause there was like a central like route to the project. And then additionally, as specifications were created, there was no way to like co-govern these things. And so what ended up happening is there was a fracturing of organizations that had spun off of the initial Linux software code base.

And so this is why today, like you see lots and lots of different distributions of Linux. And so I think that one of the problems that we're trying to solve is that fracturing problem, where we can bring people together that are interested in this project and we can allow them to have stake in the governance of the toolchain and the specification. And this way we can try and balance balance people's needs of what they're looking for from the standard. And the same way, let's say you have a standards body for browser standards.

And so that's like another example you could think about kinda like the browser wars between Internet Explorer, trying to write their own and be a player at the table, versus the Chromium team building the Chrome browser. And today, it all has consolidated to the Chrome standard and it's hard to actually get a say at the table of what goes on there as an organization who was using it. And yeah we're trying to solve that problem with this project so that let's say tomorrow, somebody says, "Hey, we really want to use Polywrap, it's a cornerstone of our technology solution, we want to become a member of the DAO." They could make a proposal to let's say gain governance rights maybe in exchange for maybe some of their token or just, an exchange for, let's say more funding or something like that. So it allows you to have an openly governed thing that that can try and balance the interests of all interested parties and limit the need for hard forking.

Nathan: That's that's a brilliant analogy into the Linux group, but the Linux foundation is actually a foundation. But they probably weren't initially started as a Linux foundation. They were just a group of devs.

Jordan: Yeah. And I think that, the process for how to get involved in the Linux foundation may not be as easy as something as like a token holder governed DAO.

Nathan: Yeah, I'd love to understand that a little bit. Can you can go in and discuss, like what's the best way to get started creating Polywrap rappers and I guess you call them Polywrappers?

Jordan: Yeah. So Kevin here actually did a ton of great work working on our documentation. And so we have a website https://docs.polywrap.io/ . You can visit that to get an intro to the project which we'd love to walk you through a bit today so that you can actually understand like what we're talking about and why this project is exciting. Kevin would you like to add anything?

Kevin: No. Just from a contributor standpoint, like if you want to contribute and build Polywrappers, join the discord and intro yourself. And we can explain the project to you and walk you through the docs and then help you get started.

Nathan: Awesome. Can you give us some promising you key use cases and some examples of some partnerships who are excited to use it?

Jordan: I just wanted to add one quick thing. Anything you can build with JavaScript today, the idea is that you can build this with a Polywrapper and it's going to have all the benefits of what Polywrappers can do. And we can describe that a bit. So the possibilities are really limitless, but I'll allow Kevin to speak on like some of the exact use cases that we're working on.

Kevin: Yeah. One of our most exciting use cases has been working with the Uniswap grants. So we received a grant for building the Uniswap Polywrapper.

And what we did with that is we built the business logic of Uniswap. So all the swap functions and we put it into this Polywrapper deployed to a decentralized endpoint, which is IPFS in this case. And we took the Uniswap interface, forked it, and replaced all their installed JavaScript as the case with Polywrappers.

What that means is like when the Uniswap interfaces launches, it's able to query the functions exposed by these Polywrappers the Uniswap Polywrapper, and it's able to use that at runtime. So that's the kind of advantages that we can get into. Yeah that's one huge use cases that we were able to do in the past few months.

Nathan: So let's go over that. Why specifically, were you interested in doing in compiling those function calls?

Jordan: It's not compiling them at runtime, it's just fetching them at runtime. The idea was to recreate the type of usability you have when building a web2 application. For example, you want to integrate Stripe's API or Twitter's API.

All you have to do is send a HTTP request to their endpoint to try and initiate a payment or to make a post. And in the world of Polywrap, what we wanted to do was allow you to simply say, I want to execute a swap on Uniswap and just send that query to the decentralized endpoint, which is in an ENS address, let's say, for example, which then resolves to an IPFS package. And so that IPFS package, it contains the WebAssembly code. So we're getting a bit into the details here, but the WebAssembly code contains all of your business logic, and that gets downloaded into your application and executed at runtime.

And so what this means for developers, if you're an app developer, what this means is that all you need to add to your application is the Polywrap client, it's just a simple JavaScript package you can add to your JavaScript app or in the future, let's say a Rust crate you can add to your Rust project and you can use that client to query any Polywrap that exists on the decentralized web. And so it allows you to use anything at run time and it doesn't bloat your application. So you can theoretically have an infinite number of web3 integrations into your application without having to bundle them all in to your app.

Nathan: That is super cool. So when you guys consider this, like what do you imagine in your infinite string of a web3 API calls that currently don't exist?

Jordan: Yeah. Thinking of all of the SDKs that exist today for web3, let's say for IPFS, they have SDKs for Ethereum, all the different smart contract protocols that exist today.

And so we expect all of those, some of them already exist as Polywrappers, and this would allow them to be used in any language. It allows them to be, fetched at runtime, so your applications can remain lightweight. A quick note on that is that this whole lightweight application thing is very important for not only like memory-constrained devices like IOT devices, but it's also important for composability and aggregation.

In web3, the whole idea is that it's an open field of innovation that can be extended upon into the future. And if you think about the DeFi aggregators that exist today, this allows them to more easily do what they're doing inside of their app. And, potentially like through the use of standard interfaces do so without having to rebuild their application.

So tomorrow when a new swapping protocol gets released or when a new staking protocol gets released the application could automatically and dynamically show those. So it, it in the same way that money Legos works on the blockchain where you have open extendability, we're bringing that same sort of extendability to the applications that display these protocols.

Nathan: Got it. Can you give me some examples of what you've considered as you're building these? Like I know there are DeFi aggregators, but what about where do you see them when they implement the Polywrappers? What do those products look like?

Kevin: So potentially if you integrated like the Uniswap Polywrapper with the Maker Polywrapper and execute a swap in your Dapp that you're building, and then take that as collateral and mint DAI with it, all using Polywrappers. That's one use case that comes up.

Jordan: Yeah. And another kind of interesting use cases thinking about extendable protocols. So let's think about a DAO protocol with different types of voting machines. You could make it so that your application automatically shows all of the new voting machines that get published to your marketplace and users can just choose them inside of the app and use them at will. And because it's all functioning off of these dynamic SDKs that get downloaded, there's no need to rebuild your application.

You deploy your application once using these standard Polywrap interfaces. And then it can automatically show all of the new functionality that gets published to the chain.

Nathan: That's super cool.

Jordan: Yeah. And one more note on that, not just for user-facing applications, but this is very important for nodes. There's a really great protocol that I got released recently called called ceramic. It was a spin out from the 3box team and it's all about decentralized data schemas.

And so schemas is for, let's say your profile or for let's say your different addresses on different blockchains and they built it in such a way where these ceramic nodes can have different extensions added. They call them streams so different stream types. And currently today, if you wanted to add a new stream type, you would have to add the JavaScript files to their node their nodes code base, and then all the nodes would have to update themselves in order to support this new stream type. But if in the future they chose to, nodes could dynamically fetch different stream type Polywrappers for handling these different types of documents. And so the nodes don't have to, upgrade themselves and redeploy themselves, they can simply just persistently run. And then as soon as a new stream handler gets deployed, they'd be able to use it. And so users would be able to use these nodes with this new type that got added. So yeah it's really important for the like building of nodes that are future-proof.

Nathan: Interesting. And where do you see this fit into the currently existing sort of bridges in the blockchain ecosystem? Like the Avalanche-Ethereum bridge, then there's Cosmos itself, being at the bridge of bridges. Where do you see this fitting into that ecosystem?

Jordan: Yeah, that's a really great point. So when you think about how a bridge works there are the smart contracts that live on the two different blockchains that you're bridging in between. And then there is the logic inside of the bridge node that knows how to handle requests and process them. And currently, our solution would basically give new capabilities to this node.

And again it would allow it to handle new types of bridges that get released. If you built a node for bridging in between xDAI and Ethereum and it used to Polywrapper and that Polywrapper had this like standard interface, you could publish a new bridge, let's say for Ethereum to polygon and these these kinds of like Polywrapper enabled bridge nodes could then just automatically fetch that and now use that inside of themselves without having to be redeployed. So yeah, like this kind of extendability when it comes to like dynamically fetching things, can be used in any place where you want to have lots of support for lots of different things. And you're not sure what new support you'll need in the future.

Nathan: So if you're moving assets between two chains, can you do that with a Polywrapper?

Jordan: Yeah, definitely. It's, it's important to remember that it's just arbitrary code that's executing inside of your app. So anything you can do with JavaScript, anything you can do with Go, Rust, any other programming language you can do it inside of a Polywrapper.

Nathan: That's interesting. But it sounds like from reading the website, that it's mostly focused on dev tools rather than the sort of interoperability jargon that we see in other ecosystems.

Jordan: Yeah. I think that it's just where we live in the stack. So it's the same way when you visit, let's say JavaScript's website they don't necessarily say build awesome NodeJS applications, right? JavaScript is a language, NodeJS is a platform for building multi-platform JavaScript applications, and then also backend services.

To answer your question where we live in the stack, we don't necessarily want to say, "Hey, use Polywrap for this specific use case." We obviously some have some ideas of what those use cases are, but that's up to our launch partners. Like we said, we're working with lots of great partners the team at Gelato, Gnosis, Fleek, WalletConnect, they all have different use cases that, some are really interested in this chain, interoperability, some are interested in the multi-platform support, they all have different interests of why Polywrap makes sense for them. And yeah, on our website, we highlight our key benefits, but we don't necessarily say, "Hey, we're a solution for bridging between blockchains," because that is a project on top of Polywrap.

Nathan: Interesting. Got it. And can you go into how people create plugins and can they be open source and what happens if plugins get deprecated? Can you go into the plugin idea within Polywrap?

Jordan: Yeah. So plug-ins is the terminology we use for using an existing SDK. So think of like an existing JavaScript SDK. We know that it's impossible to, rewrite the world of JavaScript SDKs in WebAssembly for both practical reasons and also technical reasons. And so there are still going to need to be some JavaScript packages that are used alongside the Polywrap client. As well as let's say, you're using the Polywrap client in Rust or in C#, there's native Rust or C# packages, you would want to be able to use with your Polywrappers.

And so plugins allow you to take these SDKs that live in the language of the client, so they live like in the clients host environment, and you can plug them in as if they were a WebAssembly based Polywrapper. And so it really just creates a little shim between the SDK and the client. And this allows WebAssembly to call into JavaScript let's and then, the other way around as well for these plugins and Java script to call into WebAssembly modules.

And so currently today we use plugins for things that are needed inside of the host environment. So things like networking for like HTTP and websockets and things like that. And then also things like signing with a private key for a blockchain let's say, so having a blockchain provider added into a blockchain provider. It allows you to add hosts capabilities to your Polywrap client to do things that the web assembly code may not do. And then additionally, it just helps you move faster because you may not want to rewrite something in WebAssembly. You can just use a plugin for it.

Nathan: And how many plugins currently exist? Are you guys incentivizing the plugin ecosystem through the DAO?

Jordan: Yeah, so it's important to note that like plugins are a subset, like it's plugins are not the superior way to go. It's kinda just a It's a solution to a problem, but Polywrapper is like the ideal solution that, we've created. We hope that, in the future and we're already moving towards those goals, that all of the logic for interacting with Ethereum, all the logic for interacting with IPFS and, Uniswap, all of that lives in WebAssembly, cause that's the ideal case. But for things that we can't necessarily do in WebAssembly, send HTTP requests, that lives in a plugin.

And then to answer your question of like, how are we incentivizing the creation of Polywrappers? We are currently doing this within our team and then other teams outside of the main core developers.

First off the core devs, we focus on high priority Polywrappers. And so we, we started with IPFS, ENS and Ethereum. And we also got a grant from Uniswap to build the Uniswap. And then in addition to that, we're working with other teams that are outside of the core development team to build other high priority use cases.

So like for example, like we're talking with Nathan here and his team for creating a Tezos Polywrapper. And then additionally we're talking with a few other teams for example we're talking with ChainSafe to see if they were interested in building a Polkadot Polywrapper that could be used with a substrate based chains.

And how do we incentivize this? The DAO has governance tokens. Yeah. It can give to these teams that build these Polywrappers. And so it, it just goes along with the ethos of, if you're building on Polywrap building these cool Polywrapers, we want to give you governance tokens because you have become developers of the project and you should be able to help govern the project as such.

Nathan: Awesome. Yeah. I'm excited to see ChainSafe takeoff. They've done some great work and we'd obviously love to integrate with Tezos. So that, yeah, that's exciting to hear that there are other folks working alongside us. Can you talk a bit about. Inside of plugins. Can you talk about can anyone publish them or like, how are they vetted? What happens if a bug is found in a plugin?

Jordan: Yeah. Or are you referring to, JavaScript based plugins or are you referring to Polywrappers? Written in WebAssembly?

Nathan: Let's run through both.

Jordan: Okay. So with plugins, as I said, in the case of JavaScript applications it's just a JavaScript package. The same way you would publish an NPM package you can publish an NPM package that is a Polywrap plugin. So nothing really changes there. And then as far as security and deprecation and all that stuff goes, the same rules apply there.

JavaScript is not the most secure. If one of your dependencies is compromised, it can really mess with your application and expose the user to hacks. And there's nothing we can really do there. But if you build it as a Polywrapper, which is built using WebAssembly those are actually executed in a secure environment, so those are secure packages. They can't access your application. And so you do get a enormous security benefits if you build it as a Polywrapper, as opposed to a plugin anyone can publish them, anyone who can upload files to IPFS or pin things to their ENS domain can do that to, store and name a Polywrapper. And then in order to make the concept of, deprecating functionality more easy for Polywrap developers, we're building a a Polywrap registry. And so you can think of this as similar to NPM for JavaScript packages, where you can publish different versions of your packages.

We're doing the same thing for polywrappers. And we're currently developing this now. And the idea is that you can one have them all, in a central registry, so you can search through it see what all the different factors are. And then two, we can give stronger guarantees for backwards compatibility, when you upgrade a package to a different version let's say. And so this allows you to ensure that your dependencies don't break in the future. If they publish a new version that changes the scheme on or something like that, or they, remove a function that you were dependent upon.

And we can talk more about how that actually works if you'd like, but yeah we're trying to build a much better version of NPM tailored towards the Polywrap ecosystem to help with the versioning and compatibility problems.

Nathan: Oh that's really cool. Wait, where where are you guys in the roadmap of that development?

Yeah. So we, have a really talented developer working on this. His name is Yuri. He joined the DAO I think three months ago he has been doing amazing work. And I've been collaborating with him because this had been an idea in my brain for awhile, that I had very like minimally fleshed out and he's taken it and ran with it.

So where we're at or where he has brought this so far, is that we have a completed spec. It looks really great. We are planning for an initial integration with ENS. So if you have an ENS domain, you can convert them into a Polywrap registry domain, and we have the initial contracts are written out for the version registry and what we're working on now is the addition of a Oracle solution to ensure that backwards compatibility is able to be enforced.

If you're familiar with how semantic versioning works for packages for example, 1.1.1 is an example of version number. If you change that version to 1.1.2, just doing a patch upgrade with that third number, we want to make sure that you don't actually change the schema for example, so that, that way you don't break users because patches should never change the interface. And so that's something that we're using the Oracle network for is to basically enforce the semantic version and guarantees that you should have.

That's great. That, that's really interesting. We have, do you guys have just one main dev working on it, or how do you imagine scaling.

Jordan: Yeah myself and Yuri have been collaborating on it, but he has been driving it. It's really not too much. A lot of the work comes into play with spec'ing it out.

But then when it comes to the implementation of the contracts it's straight forward. Cause it's just like a simple registry contract with a few additions on top. And then, we're definitely building on the shoulders of giants. In order for our Oracle solution we are using some existing solutions that are already out there, like for example, reading xDAI so that Oracles can they basically vote on proposals in a very very gas efficient way.

And also, like we're still we've looked at lots of different layer, two solutions out there. But currently we're looking at xDAI. And yeah it's not too much work, but we would like to scale this up in the future. The scope actually isn't that big it sounds like it's something big, but it's really not that, that rough, it's just a registry contract, a simple voting contract. And then you write like a node to fetch and validate packages.

Nathan: That's awesome. I thought it would be a much larger infrastructure. Let's talk about folks who are already node operators and those who are familiar with running staking nodes. Do you imagine that those people will be drawn to run the Polywrap node of sorts to become like a relayer in between chains. What's your thought on that?

Jordan: Yeah. Yeah, this is great that we're talking about this, cause this is a path that we're looking at moving into, but it's very unknown right now. So a big reason of why the scope is low is because our solution for the node I was talking about for verifying package upgrades, we're planning to just use a simple set of validators that are whitelisted.

So we're not going to be doing this as like a public network yet. But we're very interested in moving in that direction. And yeah we're interested in potentially using nodes to validate packages in the registry. Like I had said also, participate in source code verification, which is another piece to our puzzle that we're planning on initially doing with a set number of of waitlist validators.

And then thirdly, we're looking into solutions for persisting packages on IPFS, let's say. And so that, that way you can pay on chain, for example, you, you publish your Polywrapper to the registry and then you pay, for a year lease maybe you pay an ether or something.

And the nodes would basically make sure that it's kept online. We're looking into potentially using Pocket Network for this. There are another one of our launch partners and they've talked about wanting to support IPFS pinning with their nodes. And I think that sounds like a great idea.

So yeah there's a few things here that we may end up doing with a Polywrap node, but there's lots of solutions that already exist out there. We're definitely not trying to roll our own if we don't have.

Nathan: That's cool. I didn't realize that pocket was doing that sort of work. But that sounds like a really interesting pivot for not only for node providers, but also for people who want the larger business of verification.

Jordan: Yeah, definitely. Another group that we've talked to was the the radical team.

And they're very interested in this concept of a repository for packages and also for CICD or workflows that can run on radical repositories. And they're also interested to see what solutions we come up with for our own versioning registry to see if we can potentially collaborate there.

Yeah there's definitely lots of web3 infrastructure related projects that are either doing something that we would love to use or interested in something that, we're starting to tackle ourselves first.

Nathan: This has been great. Do you want to run over anything else for the listeners?

Kevin: Just one big project that's on our roadmap, which is the Polywrap hub. And this is just a really nice UI for people to come and explore different Polywrappers that have been made and deployed already. And also try them on the spot using the Polywrap hub. And yeah, they can deploy, they can test it out all in the same spot.

Nathan: Great. And just once again, give us the links that work. We'll obviously share them in the show notes, but give us the places that you want people to go. So that's the Discord and the GitHub?

Kevin: Yeah, definitely. We're trying to build out the community. So the priority is try and join the Discord and then coming and introducing yourselves, regardless of whether, you're a developer or a biz dev person or an artist we want to talk to you. So we'll share the link to the recording.

Jordan: Yeah. And it's our website is polywrap.io. So feel free to go there and you can find everything there. Polywrap.io.

Nathan: Great. Kevin, Jordan, thank you so much for your time, I appreciate you coming on the show.

Kevin: Yeah. Thank you.