Friday, February 15, 2019

A Primer on Augur and Veil

A Primer on Augur and Veil

A photo of Augur's UI
This article is specifically focused on binary markets and assumes a basic understanding of how Augur, 0x, and prediction markets work.

January will be a big month for Augur. Second layer Augur applications Veil launched on January 15 and Guesser started rolling itself out to select users on January 29. While Augur has had over $1.2 million staked on their platform to date, there are still multiple hurdles a user needs to jump through in order to trade on an Augur prediction market. By making the user experience easier to use, Veil and Guesser aim to increase the adoption of this technology and open up entirely new applications. For example,


Before I explain how to use Augur, I want to define some concepts that are important to understanding how Augur works.


Volume is the cumulative amount of value that has changed hands in exchange for Augur shares using Augur’s native exchange. When an Augur share is bought, the volume will increase by the amount that was paid for the share.

The volume metric displayed on Augur is not an accurate measure of a given market’s liquidity for two reasons. First, users can easily wash trade on Augur (Buy and sell a share with themselves), which artificially inflates the volume number. Second, there is a bug in the Augur software that does not divide the volume by the proper denominator.


There are two different fees that are charged on Augur: creator fees and reporting fees. Anyone can create a market on Augur, however, you need to pay a substantial gas fee to create the smart contract that operates the prediction market. To incentivize market creation, market creators can set a fee that is paid out to them — it’s called the creator fee. The reporting fee is used to pay for Augur’s oracle system. This fee is dynamically calculated every seven days and is based on both the price of REP (Augur’s native token) and the total open interest in a given prediction market.

These two fees are known as settlement fees on Augur, and they are only charged on transactions that decrease the total amount of open interest. I believe that fees will move towards an equilibrium that compensates market creators for the setup fee but does not take away significant profits from traders. Currently, there is a wide range in settlement fees for markets on Augur, which is something to be aware of before you place a trade.

Open Interest

Open interest is the total amount of ETH that has been locked up in a given prediction market. (Each Augur prediction market is its own smart contract). Open interest increases when new Augur shares are minted, and the ETH backing the original orders is locked up in the unique prediction market smart contract.

Open interest can decrease in two different ways. The first way open interest decreases is when a complete set is closed out because the open interest is used to market make against the two sell orders. When a complete set is canceled, settlement fees will be charged. In Augur’s native UI, there is no way to determine if the orders are backed by ETH or shares, which can lead to surprise settlement fee charges when placing sell orders backed by shares.

The second way that open interest can decrease is when a prediction market has been reported on. Once this event occurs, the correct share (long or short) will be worth 1 ETH. An individual can then redeem their winnings, which will decrease the open interest. The amount that a user will be able to redeem their shares for will be equal to 1 ETH minus the settlement fees.


The date that the reporting period for a given prediction market begins on. After this date, the outcome for the prediction market should be known.

Unique Properties of Augur

It is important to understand this section if you want to use Augur’s native UI. You don’t need to read this section if you want to just use Veil.

In Augur, there are two ways for an order to be filled. It can either be filled by a counterparty paying ETH — like on a traditional exchange — or by a complete Augur set being created or canceled.
An Augur set is created when a party goes to purchase shares on Augur and another party takes the opposite side of their bet. Also, the combined total of these two orders has to be equal to 1 ETH. For example, I can submit an order to buy one long share in a binary market at a price of 0.7 ETH. If there is another order to buy one short share at a price of 0.3 ETH, a complete Augur set will be formed. When a complete Augur set is formed, new long and short shares are minted and issued to the two parties in exchange for their ETH.

A complete Augur set is canceled when there are two opposite sell orders for a combined total of 1 ETH. These two orders will be matched and the respective shares will be burned. The ETH that was being escrowed by the prediction market smart contract will be paid out to the respective makers of the orders.

This means that orders can be mirrored across order books, which gives Augur and subsequent second layer relayers twice as much liquidity as traditional exchanges with the same amount of orders.

Here is an example to show how the unique properties of Augur allow a buy order for a short share at 0.3 ETH to be mirrored onto the long order book as a sell order for a long share at 0.7 ETH. A buy order for a short share at 0.3 ETH can be filled by either a sell order for a short share at 0.3 ETH or a buy order for a long share at 0.7 ETH. Both of these subsequent orders can be filled by a sell order for a long share at 0.7 ETH. This means that a sell order for a long share at 0.7 ETH can be added to the long order book, even though no one explicitly placed an order for this.

Another way to think about this is to denominate all orders in long shares with the knowledge that orders can either be backed by shares or ETH. Here is an example to illustrate this concept.
If we wanted to buy a short share for 0.3 ETH, we could submit a buy order for a short share at 0.3 ETH and a sell order for a long share at 0.7 ETH. Even if we didn’t own a long share, we could still submit a sell order for a long share at 0.7 ETH, the order would just be backed by ETH and not shares. Since we are only using the long order book, our sell order for a long share is the only order that will actually be added.

Imagine that a buy order for a long share for 0.7 ETH is submitted to the order books after we have submitted our sell order for a long share that we don’t actually own. Our orders will be matched and sent to Augur’s matching engine. The matching engine will parse the orders to determine if there is ETH or shares backing the orders. There are three possible ways that our orders are filled depending on what backs the two orders:
  • If there are shares backing both orders, which isn’t the case for our hypothetical example because we are selling a share we don’t have, a complete set is canceled because a buy order for a long share at 0.7 ETH is equivalent to a sell order for a short share at 0.3 ETH. The sell order for a long share at 0.7 ETH is matched with a sell order for a short share at 0.3 ETH and is sent to the Augur contract. These two shares are burned, open interest is decreased, and the ETH that was locked up in the contract minus the settlement fees is distributed to the two sellers.
  • If there is ETH behind both of these orders, which is the case in our hypothetical example above, a new set will be created because a sell order for a long share at 0.7 ETH is equivalent to a buy order for a short share at 0.3 ETH. The buy order for the long share at 0.7 ETH is matched with the buy order for a shart share at 0.3 ETH and sent to the Augur contract that mints a new set. The open interest would increase and the new shares would get distributed to the makers of the orders.
  • If there is a share behind one order and ETH behind the other order, which isn’t the case in our example above, one person would get ETH and the other gets the shares. No new sets are created or destroyed in this scenario.
Augur’s binary market order books work like in the example above: all orders are denominated in long shares and no orders are sent to the short order book. The smart contracts that hold these order books store information to differentiate if an order is backed by ETH or shares.
Orders can also be filled without Augur issuing or canceling sets. Augur maintains their own order book and matching engine, where investors can buy and sell their shares in exchange for ETH. The two ways that an order can be filled are abstracted away from the end user on Augur.

Augur’s Native UI


You can choose to interact with Augur directly using the Augur app and a web browser such as Chrome. Once Augur has been downloaded (This took me ~4.5 hrs), you will be able to browse all the prediction markets that have ever been created on Augur. Upon opening the Augur app in your browser, you will encounter a variety of numbers and terms.

Making a Trade


When you click on a prediction market in Augur, an order book will appear. Orders in green are buy orders and orders in red are sell orders. Remember, Augur’s native UI only shows the order book for long shares.

Purchasing a long share

When you want to purchase a long share because you think the outcome of the prediction market will occur, you can input your quantity and limit price into the respective fields. If you are a “rational” individual, you won’t want to set a limit price that is higher than the expected probability of the event happening. For example, setting a limit price of 0.7 ETH means that you think there is at most a 70% chance of this event occurring.

Checking the box “FILL ORDERS ONLY” will scan the order book for matching orders and only fill your order if a matching order currently exists on the order books. Your order will not be added to Augur’s order book. The reason that you might not want to add an order to Augur’s order books is if the odds for a prediction market could change suddenly. In the scenario where the odds change significantly, you would have to remember to cancel your order to prevent yourself from getting taken advantage of.


After you click submit, Augur will ask you to send an approval transaction to Augur’s exchange contract that allows it to move ETH on your behalf. Since every Augur prediction market is its own set of smart contracts, you will also need to send an approval transaction to Augur’s exchange contract any time you want to sell shares in a new prediction market. (You can check out this Medium article to learn more about ERC-20’s approve function.)


Once the order has been added to the order book, you will see the order under the “Open Orders” tab. You can cancel this order by clicking the “Cancel” button. Canceling an order will cost gas, however, it will free up the ETH that has been locked up in the order.

Once you have purchased your long share, you will see your share underneath your “My Positions” tab. You can sell this share by clicking on the sell tab and submitting an order for the same quantity of shares you own.

Purchasing a short share


When you want to purchase a short share using the Augur UI, you click on the sell tab. This is counterintuitive because you are placing a sell order for a long share that you don’t own, however, this is an equivalent order type to buying a short share as explained above.

The order book and its colors will stay the same when you switch to the sell tab because the UI only exposes the long order book. You can input the quantity that you would like and set a limit price. The price that you input will not actually be the price you pay. Since everything is denominated in long shares, the price you pay for the short shares is 1 ETH minus the limit price.

Here is an example of how the math for the actual cost of your short share works out. If you purchase a short share by selling a long share that you don’t own at a price of 0.8 ETH, you will be buying a short share for a price of 0.2 ETH. In more human readable terms, this price means that you think there is a greater than 20% chance this event won’t occur. In the scenario where the long outcome does occur, you lose your 0.2 ETH. However, in the scenario where the long outcome doesn’t occur, you receive 1 ETH — Settlement Fees and your profit will equal 1 ETH — Settlement Fees — 0.2 ETH.

In Augur, you can sell your shares before a prediction market has been reported on. If an event occurs that makes this prediction market less likely to occur, the price of a short share should increase to a price that is higher than when you initially purchased your share. If you choose to sell your short share before the market reported, your profit would equal Current Market Price — 0.2 ETH. As an aside, you wouldn’t be charged settlement fees here, unless the other side of your trade was backed by shares and not ETH.

Once you have purchased a short share, you will see a negative value under the “My Positions” tab, instead of a positive value. If you want to cancel your position or sell your short share, you will actually need to buy the same quantity of long shares to do so.



Veil is a second layer application on top of Augur that aims to increase the liquidity of markets on Augur and provide an easier way to interact with them. Only selected prediction markets on Augur are available to trade on Veil. I believe that the team takes into account settlement fees, reporting risk, and legal liability when deciding which prediction markets to list. Right now, Veil is not available to residents in the US or any of the OFAC countries.

Differences between Veil and Augur’s UI

One of the main differences between Veil and Augur’s UI is that Veil runs a matching model relayer on the 0x protocol. This makes trading Augur shares faster and easier than if you were to use Augur’s native exchange, however, liquidity is not pooled between Augur’s order books and Veil’s order books. For the services that Veil provides, they charge a 1% fee on all orders.

Second, Veil offers instant settlement for Augur prediction markets. It can take over 27 days for a market on Augur to finish reporting, which means that users won’t have access to their winnings until the reporting process has finished. Veil will let users redeem their winning shares before the reporting period has finished for a 1% fee.

Lastly, Veil is a company incorporated in the Cayman Islands, whereas Augur is an open source project that is currently being maintained by the non-profit Forecast Foundation.
I am going to define a few of the term’s shown on Veil’s trading UI below and explain how they differ than the ones in Augur’s UI.


Volume on Veil represents the same metric as on Augur’s UI. Veil’s volume metric will usually state a different amount than the volume metric that Augur’s UI reports because Veil operates their own order book and calculates volume based on this.

Last trade value

For binary markets, the last trade value shows the most recent price that a long share was purchased at. If the last trade value was .4 ETH, this means that the market thinks there is a 40% chance of the event occurring.

Purchasing a long share

To make a trade on Veil, you have to create an account and whitelist your Ethereum address. You also need to wrap your ETH into Veil Ether. Veil Ether has a function called depositAndApprove that lets users wrap their ETH into an ERC-20 token like WETH while also approving the 0x Exchange contract to transfer ETH on the msg.sender’s behalf. (This removes the approval transaction that we encountered when placing an order via Augur’s UI).


To purchase long shares, you can click on the “Long” tab. Unlike in Augur’s UI, Veil shows the order books for both long and short shares. You will notice that the sell side of both the long and short order books are always shown, independent of if you are buying long or short shares. Veil also offers a pro UI that looks like a more traditional exchange order book with a depth chart.

Veil's Pro UI

On the long and short tabs, Veil makes the current probabilities more human readable. For the order book above, you would want to buy a short share if you thought there was less than an 83% of the event occurring and you would buy a long share if you thought there was above a 92.5% of the event occurring.

Veil also includes a payout gauge, which shows the max gain and loss that you could encounter for a given trade. To learn more about how the economics of Augur prediction markets work, check out the Veil team’s article here.

There a few important differences between Veil’s user experience and Augur’s user experience. Veil is using a matching relayer model, which means that they use an off-chain order book. This means that the user can submit an order or cancel an order without having to pay gas costs. (When you place an order, Metamask will ask you to sign the order transaction, but you will not be charged any gas.) Veil is also using the 0x order format, which means investor’s do not lock up their ETH when they submit a transaction.

Purchasing a short share


Purchasing a short order on Veil is much more intuitive than on Augur’s UI. To purchase a short share, you can click on the “Short” tab. Similar to the order flow for purchasing a long share, you can enter the price per share and see potential payouts for your trade. Again, you don’t need to do any complex calculations to actually determine your costs when purchasing a short order. The price per share that you enter will be the price that you end up paying. This is because Veil denominates orders in both long and short shares.

When you want to sell your short share, you can easily sell your short share by switching to the “Sell Shares” tab.


Veil is a much better option than Augur’s UI for users who want to be able to submit and cancel orders freely, use Augur without waiting 4.5 hours to download the application and redeem their winnings instantly.

I am excited about Augur because it allows anyone to create a financial market for anything. There are two reasons that I believe this is important. First, financial markets are an ingenious invention that incentivizes individuals to work towards a common goal, and these have traditionally only been applied to a small subset of tasks. I am very excited to see what will happen when financial markets are created for tasks such as volunteer efforts or predictions. Secondly, more diverse financial markets will allow individuals to hedge their risk in many more ways than what is currently available. While it may be years before these goals are realized for mainstream consumers, the Augur ecosystem is actively growing and the user experience is improving.

I want to thank @micah, @aaaaaaaaaaaaa, @ryanb, and @joeykrug from the Augur Discord group, Paul and Graham from Veil, and Justin at CoinList for answering questions that I had and reviewing drafts of this article.

If I have made a mistake in this article, please comment below. If you are still confused about how Augur works, feel free to message me on Twitter or comment below, and I will try my best to help you.

Tuesday, August 21, 2018

A Systems Design Analysis of Ethereum

            On July 30, 2015 Vitalik Buterin, a cryptography researcher, published the Ethereum White Paper. The Ethereum White Paper outlined a blockchain implementation that was Turing complete and stateful, two attributes that the first blockchain implementation Bitcoin did not have.[1] In this paper, I will provide an overview of Ethereum and blockchain technology, as well as a system design analysis of Ethereum.
            The difference between traditional distributed database systems and blockchains is that participants in a blockchain network are assumed to be bad actors, whereas distributed database systems require trust amongst the nodes. Attempts to develop Byzantine Fault Tolerant distributed database systems such as Byzantine Paxos require multi-stage messaging processes that are not scalable for internet size deployment.[2]
At the most basic level, a blockchain is a linked list data structure. In Ethereum, each node within this list has a variety of parameters, including the hash of the previous node in the list and three tree data structures.[3] These trees store the state of the Ethereum blockchain, a list of transactions, and a receipt log. The state of Ethereum is made up of accounts: externally owned accounts and internal accounts known as smart contracts. External accounts hold balances of the native cryptocurrency Ether, while smart contracts hold code. To add code to a smart contract or execute a contract’s function, a transaction that originates from an external account is sent. When a transaction is processed, a state change occurs. The third tree data structure stores event logs. Just decentralized databases have logs, Ethereum smart contracts will emit events that write to a log after performing some action.
            In order to maintain this data structure in a decentralized manner, an incentive scheme for validators is required. The validators of transactions in Ethereum are known as miners. When a transaction is sent to the network, it is stored in a pool of unconfirmed transactions. Miners will choose a set of transactions from this pool to validate. Validating a transaction involves running each transaction and ensuring that the transaction adheres to the rules of the Ethereum protocol. After validation, the miners will hash a group of transactions together into a block. If the resulting hash is lower than a required level, the miner can share this block with the rest of the network.
This process of validation is known as Proof-of-Work. Requiring miners to produce a hash that is lower than a certain value ensures network security because miners must spend computational power and electricity to produce these calculations. Miners will conduct these actions in exchange for the opportunity to earn a monetary reward.
            Other nodes in the network will run these shared transactions, which is a non-computationally expensive process. The transactions will either be accepted, if they adhere to Ethereum’s protocol, or rejected. Other nodes in the network have an incentive to accept the first valid transactions they receive because they will be interested in building off them as the next node in the linked list.
            Another way to think of Ethereum is as an incentive scheme. Simply put, miners who validate transactions are awarded a reward by validating transactions. Some miners may attempt to publish transactions that fraudulent i.e. sending a transaction from an account that does not have a large enough balance. When this transaction is propagated throughout the network, other nodes will see that the transaction does not abide by the Ethereum protocol and will not build off of it in their linked list. These malicious transactions will end up being a waste of work for the malicious miner because they will not receive a reward for these transactions.
System Analysis: Forward Compatibility Vs Backward Compatibility

            With any distributed systems, versioning and managing upgrades can be difficult. Since Ethereum is a decentralized network, software upgrades are very difficult. There are two different ways to upgrade blockchain software: soft forks and hard forks.
Soft forks occur when backward compatible upgrades are made to the protocol. This means that the new protocol is more restricted than the legacy protocol.[4] Nodes that have upgraded their software will see that the new blocks conform to the new consensus protocol. Old nodes will also see that these new blocks conform to the legacy protocol since the new protocol is a subset of the old protocol. If the majority of nodes have upgraded to the new software and an old miner submits a block, this block will be viewed as invalid by the network because it does not adhere to the stricter protocol. Miners who have not upgraded their software will have an economic incentive to do so, as their blocks will not be accepted by the network.[5]
Hard forks expand the existing protocol rules. During a hard fork, nodes that upgrade their software and nodes that do not upgrade their software will operate on two different chains. Legacy nodes will view the newly mined blocks as invalid because they correlate to a different consensus protocol and vice versa.
Soft forks are viewed as less dangerous to a network, as a chain split is less probable. In the case that a majority of miners do not upgrade their software and a majority of nodes do, a chain split will occur. In this situation, nodes will most likely be forced to downgrade their software.
Within the cryptocurrency community, soft forks have developed mixed opinions. Individuals such as Mike Hearn, a Bitcoin core developer, view soft forks as trickery. He argues that the forward compatibility of soft forks really just tricks legacy nodes into thinking updated transactions still follow the legacy protocol when they are actually different in nature.[6]
Vitalik Buterin and Jeff Garzik, a Bitcoin core developer, believe that soft forks do not let users opt into the new consensus rules.[7] In a blog post, Vitalik states, “[W]hat matters is not whether or not individual users have to perform the simple bureaucratic step of clicking a “download” button, but rather whether or not the user is coerced into accepting a change in protocol rules that they would rather not accept.”[8] Vitalik argues that within the blockchain space, individuals think that soft forks are less coercive because the individual does not need to actively upgrade their software. Instead, Vitalik argues that by being passive in a soft fork an individual accepts the new protocol changes, which they may be unaware of. Bitcoin core developer Eric Lombrozo offers a differing opinion. He states that the Bitcoin Improvement Process (BIP) that implements soft forks is community driven and individuals always have the choice to run different software if they are not pleased with the current iteration of the software.[9]
Historically, hard forks have been more contentious to implement than soft forks. This is due to the fact that chain splits diminish the network effects of a particular blockchain and hard forks tend to evolve around contested governance and philosophical issues. For example, Ethereum conducted a hard fork in the summer of 2016 after a hacker exploited a bug in an Ethereum application that took $50 million dollars from users.[10] This hard fork resulted in a chain split producing Ethereum and Ethereum Classic, the blockchain that did not return the lost funds to users. This hard fork decision raised questions amongst community members about the immutability of blockchain systems and the governance processes for decentralized networks.
Andrew Poelstra, a Bitcoin developer and employee at Blockstream, wrote an email thread arguing that hard forks increase the cost to maintain a node, which will lead to centralization. The reasoning for this statement is that hard forks will require individuals to actively monitor their software, which they might choose to outsource.[11] Others argue that forking allows a network to iterate faster and solve governance issues that a subsection of the community may have with the current network.[12]
 Apart from the diminishing network effects of splitting a network and determining who gets to use the original network name, legacy blocks remain valid on the new hard forked chain. This opens up the Replay Attack Problem. A replay attack occurs when an individual sends a transaction on the original chain at the time of a hard fork. Since this transaction will be valid with respect to both chains, this transaction can be broadcast to the new chain. The original sender may not know about this transaction that has been maliciously broadcast, which will result in the same state change.[13]


            Right now, each type of account is identified by a unique 40-character hexadecimal representation. In an attempt to make these account names human readable and easier to interact with, a group of developers has begun working on the Ethereum Name Service (ENS). The ENS acts as a decentralized DNS system.
The ENS is made up of three Ethereum smart contracts: the registrar, the ENS registry, and the resolver. Nick Johnson, the lead engineer for the ENS system, stated that by breaking up the services of a DNS into three simple pieces, upgradability and forward compatibility are easier to maintain throughout the ENS.[14]
The ENS registry is a contract that maps the hash of a name to the name’s owner’s address and the address of the specified resolution contract. The registry will return the address of the resolution contract, which will then return the address of the object the owner has mapped the given name too. The registrar is responsible for assigning owners to top-level domain names.[15] (Currently, a multisignature account of the lead developers is the owner of the .eth TLD within the ENS registry contract.)
            The current implementation of the ENS is very primitive. For example, names can only be registered under the .eth TLD, and no name shorter than 7 characters can be registered. This is because there is no effective dispute resolution system for names within the registrar contract. J. Maurelian, an ENS contributor, wrote in a Medium article, “Who should control a name like microsoft.eth bankofamerica.eth, or vitalik.eth? How should a registrar allocate these names?” 15 The developers have effectively kicked the can down the road by waiting until a full registrar with dispute resolution has been implemented so more desirable names can be owned.
Within the ENS registry, a hash of the desired name is registered to the corresponding owner. This allows any type of foreign characters and upper and lowercase characters to be used in a .eth name.[16] Similar to the DNS, owners of a given .eth namespace are allowed to operate on any subdomains of that name.[17]

Compiler Versioning:
            All Solidity code, the programming language for Ethereum, requires that the source code includes a compiler version. As explained in the Solidity specs, “Source files can (and should) be annotated with a so-called version pragma to reject being compiled with future compiler versions that might introduce incompatible changes.”[18] The most current version of the compiler is 0.4.12. Any code written with a compiler version 0.4.0 or greater will be compilable with 0.4.12. When version 0.5.x is introduced, code with earlier compiler versions will not remain forward compatible.
From a development perspective, compiler versioning is inconvenient because individuals have to have multiple versions of the compiler installed. Legacy source code may not remain supported far into the future, as individuals will remove those compilers from their systems. While researching this paper, I came across a Github comment section where individuals wanted the compiler version to be included in the binary of the Solidity code. The rationale is that any individual would then be able to validate the given smart contract code.[19]
            While Ethereum is a truly decentralized database, many of the same system design problems of centralized systems exist within its implementation. While it is still in the early days for blockchain implementations, blockchains enable systems to be built that do not require permission, centralized points of control or trust.