GAIA read, write, and throttling specifications?


#1

For polling applications, I am wondering what the limitations of GAIA are for the number of read (putfile) and write (getfile) requests? Is there any request limiting that occurs when a threshold is reached? Are there any best practices recommended for repeated reads or writes from the same endpoint–i.e. caching the endpoint returned from the zonefile for a session?

I imagine these limitations would be those of the underlying storage–i.e. azure or dropbox, but I’m also wondering if there are limits in accessing zonefiles?


#2

The public Gaia hub that Blockstack PBC gives users access to for free does indeed throttle reads and writes. However, if you run your own Gaia hub (instructions), you are only limited by how fast you can load and store data over the network.

You load and store your Blockstack data via HTTP, so you can use any/all HTTP-based tactics to accelerate reads. The Gaia hub is not on the read path—when you read data from the Gaia hub’s designated read URL, it’s hitting some HTTP-speaking process (e.g. Dropbox, an S3 bucket, an Azure bucket, an HTTP server, etc.). This means you can use any “standard” tactic for accelerating reads—you can use a CDN to cache reads, you can use memcache on the Gaia hub to accelerate origin server reads, etc.

The Gaia node runs on the write path to authenticate writes and stream data to a storage system “driver”. The performance you’ll see will be either the raw performance of the storage system, or the raw performance of the Gaia hub’s ability to stream the write—whichever one is slower (the Gaia hub is written in node.js). If you find that your write path is bound by your Gaia hub, you can stand up multiple instances and use a TCP or HTTP load-balancer to distribute writes across them.

Zone files are loaded from a Blockstack Core node, not Gaia. Each Blockstack Core node has a full replica of all zone files in existence, so if you run a node on localhost, your zone file queries will be near-instantaneous. However, most clients are expected to read from a public Core node. You can read more about how to deploy a public Core node here (these instructions detail how we deploy core.blockstack.org).