Multi-PPS: Dissolving mining pool centralization

The Bitcoinsphere is abuzz with news of the GHash.io mining pool exceeding 50% of the Bitcoin network hashrate, which is of course in contrast with Bitcoin’s decentralized vision.

While it is indeed worrisome, I personally believe it is not something to panic about, as the underlying causes are solvable and we should not see such a phenomenon recur going forward.

The current centralization is the result of two distinct causes. One is the concentration of mining within a few large mining farms, often controlled by those who manufacture mining ASIC chips. These either take the mining risk themselves or forward it by offering mining contracts to the public. This is made possible by the economies of scale in building machines and operating them that large entities enjoy. This is indeed a problem, however, I believe this will be alleviated somewhat as SHA-256 computation becomes more of a commodity, as I claimed in ASIC will not Centralize Bitcoin Mining.

The second cause is that even individual miners who operate their own machine independently, use these machines in conjunction with large mining pools. The reason is, that with all classical mining pool reward methods, the larger a pool is, the better performance it can offer its miners in terms of the tradeoff between variance, fees and maturity time. A large pool will be more lucrative and hence attract more miners and become even larger, with the ultimate limit being consolidation within one pool.

Fortunately, I believe the latter cause is easily solvable. All it takes is using a reward framework that does not lead to “the rich get richer” effect. P2pool is often cited as a possible such framework, but while it has its merits, I do not believe it has what it takes to offer a sustainable solution, mostly due to the difficulty of small miners to use it.

Starting with p2pool as a basis, thinking what can be done with it and following it to its logical conclusion, I arrived at Multi-PPS, which I wrote about a year ago. Its basic premise is that miners should mine in multiple pools simultaneously, in proportion to each pool’s strength. Using this framework has two important features:

  1. The miner enjoys performance that is equivalent to that of a pool with a combined size of all pools he uses together.
  2. The stable equilibrium is not consolidation in one pool, but rather, maintaining a distribution between many pools according to the merits of each.

Of course, implementing this is nontrivial, and there are some technical performance issues to consider; however, the numbers do look promising, and the effort in implementing it is small relative to the potential gain. More details are available in the linked post.

Because this is such a hotly debated topic now and this proposed solution receives too little attention for my taste, I have decided to choose it as the topic for my technical talk in the upcoming Inside Bitcoin Tel Aviv Conference at the end of July.

Mtgox post mortem

(I’m assuming the rumors of Mtgox’s demise are not exaggerated, though it’s too early to tell).

It can currently be expected that whoever had money stored in Mtgox, will not see it again.

I am truly sorry for all those who have lost their money this way, certainly if this is a significant amount or even the entirety of their bitcoins.

But I also believe that people should take responsibility for their decisions and not expect too much sympathy if they make mistakes that cause them harm. I, too, have done nonsensical things which cost me a lot of money, and I try not to blame for them anything or anyone but myself.

Holding a significant amount of bitcoins on Mtgox was a mistake. Not just any mistake, but an antithesis to a core idea of Bitcoin. Bitcoin is about being decentralized and not having to trust anyone else to use. So what is the point with trusting all of your money with someone else?

There is a simple rule for trust – weigh what the other side has to gain from embezzlement, and what he has to lose. Anyone holding quantities of money for many people has a very strong incentive to get up and run, even if he has a successful business. 4 years in the market is really not enough time to build a reputation and trust that will justify deposits in the amounts Mtgox had.

And it’s not like Mtgox had a clear track record in these years. They’ve had lots of hacks and a variety of problems, and for almost a year it’s been nearly impossible to withdraw USD and there have been talks about insolvency.

Perhaps in a few years there will be a service reliable enough for the simple individual to trust with their money (preferably with a multisig arrangement or suchlike). But for now we are all early adopters and we should make the effort to learn how to store our bitcoins securely. It’s not trivial but not rocket science, and there are guides such as my previous post about paper wallets. Even with some basic methods you can reach a high level of surety. We hear all the time about people losing money they trusted with someone else, but despite all the talk of hackers we rarely hear of people whose own wallets were hacked. One can also divide his funds between different places (one of which could have been Mtgox) so that if anything happens one would not remain with nothing.

Back to the subject at hand. There are two problems with holding bitcoins on Mtgox. One is that an Mtgox wallet is not really designed for everyday use, so whoever has his money there is in fact stating that he has no interest in using Bitcoin but sees it purely as a speculative investment. This is not a problem in itself – it is legitimate to believe in Bitcoin and to want to profit from this belief. But part of this belief should be a belief in using Bitcoin in the way it was designed. And putting all of your bitcoins with someone without sufficient reputation and without layers of cryptographic or regulatory protection is not the way Bitcoin was meant to be used. So in essence, whoever has put his money with Mtgox is looking to profit from a belief but is not really acting on this belief. (Either that, or he doesn’t really believe in Bitcoin and simply wants to ride a trend, which is worse.)

For a long time people proficient with Bitcoin have been recommending not to hold funds in shared wallets such as Mtgox. I point that out not for gloating or a childish “told you so”, but simply to clarify that this is not hindsight, putting money in Mtgox is problematic and one could have known it was problematic. This must be taken in consideration to understand that the problem is not with Bitcoin, and to learn from it how to be more careful in the future.

I should clarify that exchanges are necessary, but this is what they are, exchanges. People who want to invest in Bitcoin, should buy and get them out of the exchange. People who want to day trade, should weight the profits of trading against the risks of holding money there.

The summary of the Mtgox issue is that (probably) an exchange that has been struggling for a long time is shutting down, and people who acted against the spirit of Bitcoin lost from it. The implications for Bitcoin are not major, beyond a self-fulfilling prophecy – people think this has implications for Bitcoin, and that itself harms Bitcoin.

I hope we all learn something from this story, and that despite the pain we will not stop believing in Bitcoin. It is not the currency which has failed us, it is us who have failed the currency.

How to create a reasonably secure Bitcoin paper wallet

tl;dr: If you don’t want the explanations, just follow the bolded instructions.

Some of us want to store a sizable amount of bitcoins as a long-term investment or savings. We need a way to store our keys (the pieces of data that give us, and only us, access to our coins) that will be safe from both loss and theft; but we do not care about them being easily usable on a daily basis. A great way to do this is paper wallets – storing our keys on printed pieces of paper.

Following this tutorial will allow you to create your very own paper wallet.

1. Go to https://raw.github.com/pointbiz/bitaddress.org/master/bitaddress.org.html. This is essentially a program that allows you to create keys (and corresponding Bitcoin addresses) in an easily printable form. Ignore the text you see when visiting the page.

2. In your browser’s menu, choose “Save page as” to save this file somewhere on your computer. All browsers have an option to do this, even if differently named. if you can’t find it try simply pressing Ctrl+S.

3. Disconnect your computer from the internet. This is very important. Your secure, offline paper wallet will be of no use if the generated keys are sent to an attacker by malicious software through an internet connection.

4. Open the file bitaddress.org.html you saved earlier (by double-clicking it). You’re not going to get your keys from the website’s server – rather, the page you’ve downloaded contains the program needed to generate the keys locally on your computer, using your browser. For this, it is best to download a copy of the page. Preferably, you should disable any addons running on your browser, for example, by running in incognito mode.

5. Go to the “Bulk Wallet” tab. The site contains many different options for generating paper wallets, but I find this one to be the best. It’s no-nonsense and allows you to print many addresses at once.

6. Choose the number of addresses you wish to print (e.g., 50), fill it in “Rows to generate”, and click “Generate”. Use as many as you can fit in a printed page in big enough letters to be easily readable.

7. Connect a printer with a USB cable. Of course you will need a printer for this. It’s safest to disconnect the computer entirely from any network, so this leaves USB for the connection. (I’m assuming you’re not using a parallel port for this…)

8. Click the page’s “Print” button to print several copies of a paper with the keys and corresponding addresses.

9. Create a file with the Bitcoin addresses (the shorter strings starting with “1”).

10. Make sure you have not saved the private keys (the longer strings starting with “5”, “K” or “L”) anywhere on the computer.

11. Clear your browser cache. This will decrease the chance that your browser keeps the keys. Instructions on how to do this on various browsers is available at http://www.wikihow.com/Clear-Your-Browser%27s-Cache.

12. Reset the computer. This will prevent malware from sending keys it has stored in the computer’s memory.

13. Reconnect the computer to the internet. It is safe to do so now with any trace of the keys erased.

14. Store the paper copies in multiple secure locations – a safe in your house, a relative, a safety deposit box, etc. Keep in mind that if a thief that understands Bitcoin gets his hand on one of the copies, the coins are gone. If all of the papers are lost or damaged to the point of illegibility, the coins are gone.

15. Send the bitcoins you’re saving to addresses in your paper wallet. You could send all of them to one address, but there are several advantages not to store too much on a single address. Split the coins however you see fit. You can always add more coins to any of the addresses. You can save most of the addresses you printed for future use. Keep in mind – once you send from an address, it is no longer secure and you will need to store your funds in the other addresses.

16. Eventually, you’ll want to actually use the funds stored on the paper wallet. That’s a problem for future you, but it is still useful to try this out so you know you can. Most clients have a feature for “importing” a private key – including Bitcoin-QT, blockchain.info and some hosted wallets. Try this option in your client of choice. The most secure method is to import the key into an offline wallet – for example, using Armory. This is fairly complicated now, but hopefully by the time you want to do it, the software and hardware to make it easier will exist.

Extra credit: As the title implies, this procedure will allow you to create a reasonably secure paper wallet. I’ve avoided some cautionary measures that I deemed would deter most users – perfect is the enemy of the good. If you do, however, want a more secure wallet, here are some more things you could do:

1. Make sure bitaddress.org is what it should be. In theory, the website from which you download the generation software could be hacked and replaced with malicious content. A simple way to protect against this is to search the web for mentions of “bitaddress was hacked”. A harder one is to compare the webpage’s SHA1 hash against the digitally signed hash.

2. Use a Linux live CD. Linux is less vulnerable to malware than Windows. This is true regardless of whether the reason is Linux being inherently more secure or hackers not bothering writing malware for it. If you don’t already use Linux, you don’t even have to install it – you can run it directly from the installation disc. For example, you can use Ubuntu for this – burn an Ubuntu live CD, and follow the procedure after booting from it.

3. Use a multi-factor system, such as multisig, SSSS or key encryption. You can encrypt your keys before printing them, so that stealing the paper will be useless without the password. However, a password strong enough to offer meaningful protection will be too hard to remember – you’d probably want to write it down; so this is really a primitive version of SSSS. Shamir’s Secret Sharing Scheme allows you to split the keys into n pieces, so that m pieces are required to recover the key. You can place the pieces in different places knowing that several pieces will need to be stolen to compromise the coins. The security of this, however, still relies on the computer in which the key is generated and split and on which it is combined and used. Using a multi-signature address is more secure, but also more complicated.

4. Don’t trust your printer – printers have memory which, combined with malicious firmware, could send their copies to an attacker once connected to the network. I don’t think this is a very serious risk, and I don’t know of countermeasures other than forgoing the printer and hand-writing the keys.

Additional tips:

1. Not all fonts are created equal. In some it is difficult to tell capital letters from small ones, which may create problems when you try to import the keys. Choose a clear font.

2. Consider laminating the papers, or using similar measures to protect them from damage.

3. bitaddress.org also allows creating QR codes for the keys. This is more space-consuming, but allows easier import.

Disclaimer: Use at your own risk! Even with the best security practices there can be failures; and these suggestions aim to balance security with accessibility rather than maximize security. I offer absolutely no warranty or guarantee and will take absolutely no responsibility if this guide leads, either directly or indirectly, to loss of coins or any other harm.

Naive views of financial markets

When I was a child, I did not really understand stock markets.

Sure enough, I knew that every corporation had stock; I knew that the stock price could go up and down; I knew you could buy stocks and profit or lose accordingly. But I thought these were all just numbers going up and down randomly. I did not know what caused these changes, or indeed, that they were caused by anything at all.

I recall one day when my father referred to the CEO of some company as being successful, seeing that he managed to bring the company’s stock price up. I was confused. Stock prices just behave randomly, right? How is the price going up any indication of that person’s performance?

As I grew up, I have learned that the goal of a corporation is to generate profit; I’ve learned that the company shares these profits with shareholders in the form of dividends; that people are willing to pay for the stock in order to enjoy these dividends, either directly or indirectly. The greater the profit potential of a company, the higher the price people will be willing to pay for its stock. If the stock price goes up, it is because the company is doing something right, and by extension, so do the people in it.

It surprises and saddens me that these days, “professional” economists hold the same naive views about financial markets that I did as a child, at least as far as their criticism of Bitcoin goes. They talk about how the exchange rate of Bitcoin goes up and down wildly without representing anything; how it is stupid to buy bitcoins; how people who want to gamble are better off in the casino or the stock market.

What they fail to mention is that Bitcoin is the world’s first decentralized digital currency. It is like the money we know, only better (in about a dozen ways). In the same way that companies generate profit, currencies facilitate commerce. The better the currency is at its job, the more it will be in demand, and the higher the price people will be willing to pay for it. And Bitcoin has what it takes to be great at its job.

For sure, there is a lot of uncertainty and speculation. If you look at the price of any traded asset, Bitcoin included, in a short enough time scale, it will be indistinguishable from Brownian motion. The short-term fluctuations are noise. But the long-term upward trend from 0 to the $700 range is the signal. It means that Bitcoin is getting better and better at its job, thanks to the brilliance and hard work of its inventor and supporters.

Bitcoin is not about the price going up, that is just a natural side effect of doing well what it is about. People who profited from it are, for the most part, not people who gambled and won – but people who had the foresight to see something big is happening, spent the time to learn what this is, understood what Bitcoin is about, believed that it would be successful, and were right. In so doing, they helped Bitcoin become what it is today, and for that they are fairly rewarded.

So to all the investment advisors that hold the financial worldview of a 10 year old, my message is – please, grow up.

Hashrate amplification attacks

A lot of people asked me about the recent paper by Ittay Eyal and Emin Gun Sirer, titled Majority is not Enough: Bitcoin Mining is Vulnerable.

In this paper they describe a hashrate amplification attack – a way to find a greater percentage of the total valid blocks in the network than an attacker’s portion of the hashrate would indicate, potentially leading to dire consequences.

I really wish I had the time right now to examine this paper thoroughly. It’s a topic that interests me, I want to be up to date on Bitcoin research, and I’d be able to comment much more intelligently if I did. Unfortunately, as is often the case with me, I am wholly occupied right now with previous commitments and barely have a minute to spare.

But precisely because of this, I want to save some time by writing down a single response I can direct people to. And I believe I have enough understanding of the issue to say this:

  1. The basic idea of the attack is not new; it can be traced at least 3 years back. Nobody seemed to panic about it back then, and no reason we should start now. I did not participate in the original discussion but have read it with interest, and mentioned it whenever the topic came up. I had half a mind to conduct a more thorough research of it myself, but could never allocate the time.
  2. They’re not the only ones to perform a detailed analysis of the attack. Over the past few months, a young Israeli researcher called Lear Bahack has independently studied the same issues. As it happened, one day before Eyal’s and Sirer’s paper was published, I sat down with Lear to discuss his results. My impression is that his research is more detailed, accurate, and thought out. He unfortunately was beat to publishing the results, but we will soon see him write about his innovations.
  3. According to Lear, the attack is not nearly as easy to carry out as Eyal and Sirer would have us believe, in particular due to unrealistic assumptions about the network topology and the operation of mining pools. There is no real danger in the near future.
  4. He also suggested a protocol change that could remove this vulnerability, which we should hear about soon.
  5. The paper title and the way they write about it elsewhere is alarmist and self-centered. They give the impression they feel they’re the first ever to find a vulnerability with Bitcoin, and try to sow panic with talks about how Bitcoin is broken. I’ll take their word for them honestly being motivated by the desire to get the potential attack thwarted before any damage is done (EDIT: I did until I saw this), but it certainly looks like they’re just after the publicity, maybe even seeking profit from price manipulation.
    It’s not the first time we see alarmist headlines inspired by academic papers. It happened with the transaction graph work of Adi Shamir et al., and with the red balloons work of Aviv Zohar et al. But in those cases the researcher gave a neutral description and the media blew it out of all proportion. It is unfortunate that in this case, the sensationalism comes from the researchers themselves.

All this notwithstanding, we should thank Ittay Eyal and Emin Gun Sirer for their work on this interesting and potentially important topic. Stay tuned for further developments.

Updates:

Lear has published a sample of his results in this forum post, and also put up his paper (work in progress) at http://arxiv.org/pdf/1312.7013.pdf.

ASIC will not centralize Bitcoin mining

Some claim that Bitcoin mining is doomed to be concentrated in the hands of a few large mining corporations, and that the advent of ASIC mining is the culprit.

I disagree.

Well, I don’t know for a fact that this will not happen. However there are several factors in play that could prevent this scenario – a scenario which is undesirable, because the more centralized mining is, the more likely it is that a majority of hashrate would collude in an attack against the Bitcoin network.

 

The cited reason for centralization is that large companies enjoy economies of scale in mining. These companies, combined, will scale up their operation until the difficulty has risen so much that mining is only marginally profitable. Since hobbyist miners are presumably less efficient, at that point mining will be a loss to hobbyist miners, so they will be forced out of the market.

An analogy is sometime given with gold mining. Unlike the old days of the California gold rush, it is no longer possible for individuals to mine gold. Gold is only abundant in specific locations, and requires a complex mining operation to extract. The barrier of entry to obtaining suitable land and setting up a mine is just too high.

However, the situation with Bitcoin is much different, because of its extremely parallel nature. A gold mine will no longer work if you build a miniature version of it, so a person cannot run a small gold mine at home. In contrast, a Bitcoin mining farm is basically a huge number of copies of a single unit doing a very simple computation. Even a small chip measuring an inch across would be composed of many such units. So mining can definitely be scaled down to a level where everyone can run a Bitcoin miner at home. My first contention is that, since a large farm is basically multiple instances of a basic unit, the economies of scale that can be obtained are fairly limited.

This refers to the operation of the devices. There is still a big barrier of entry to actually manufacturing the devices; however, I do not see this as much cause for concern. Because Bitcoin mining requires a single, simple computation, designing ASIC to carry it out will require relatively little R&D costs, when compared with more complex circuits such as CPUs and GPUs. Because of this we should expect many manufacturers of such chips, enough to ensure one would cater to the hobbyist market – if they are willing to pay for it.

Assuming an equilibrium where the potential mining revenue is mostly known, what anyone – whether a professional entity or a profit-seeking hobbyist – would pay for a device, is the profit he expects to receive from it in its lifetime. And since the revenue is essentially the same for all, what differs is the cost of operation. For there to be a significant hobbyist market, there just need to be enough hobbyist with cost of operation lower than that of the large corporations.

 

One thing going for the hobbyists is that they may have existing, underutilized infrastructure that can be leveraged into supporting a mining operation. A large-scale company would need to set up an infrastructure specifically for its operation, while a hobbyist may have unused physical space in his residence, a power grid connection with spare capacity, or a computing device he could use to assign work to the mining devices. For example, if he is using a desktop computer (unlike some, I don’t foresee the death of those anytime soon), he may have a spare PCI express slot to which he could plug a card, feeding off the spare capacity of the PSU (it may be also possible to design it to run only when the other power-hungry components are idle, which is most of the time).

But much more importantly, It can be expected that in the long term, the major cost of mining will not be capital expenditure, but power. Therefore, anyone who can get cheaper power will have a huge advantage. A hobbyist might have an arrangement of getting free electricity from his host, effectively “leeching” power from someone else – this practice may be frowned upon, but it will happen, and only at small scale. But the crux is hobbyists living in colder countries, who would otherwise use a resistive space heater to warm up (less so, those who would use heat pumps or furnaces). Any power spent on a mining device is exactly deducted from what they would have to spend on their space heater, so for them the power really is free.

Many other factors are in play – mining is inherently risky, and hobbyists might be less averse to this risk than professionals, or maybe some of them will have a relatively higher estimation of expected profit. Hobbyists might have reasons to mine other than direct profit, which could also tip the scale in their favor.

Even if there are not enough hobbyists with favorable conditions, we should also consider professional companies large enough to enjoy some economy of scale, yet small enough to be more lean and efficient than the large companies. And again, if these companies are in the market for mining devices, some manufacturer will take their money. Many such companies can fit in the global Bitcoin mining market, and while this is not the same as the ideal decentralized vision, it is still a good approximation.

 

In light of all of this, I do not worry that the economics of obtaining and operating mining equipment will be a cause for harmful centralization of mining.

Welcome to the Fiery Spinning Sword

Welcome.

I have finally done what I should have done a long time ago – starting my own blog. I imagined the concept for years, but never had the trigger or the skills to do anything about it.

Over the past two and a half years, as part of my involvement with Bitcoin. I participated in online discussions. A lot. Once in a while, I would even say something intelligent and meaningful; something that deserved to be fleshed out, placed in a more prominent location, and saved for posterity.

Whenever this occurs in the future, I will use this blog for it. Naturally, I intend to write about topics that interest me, such as Bitcoin, mathematics, and myself.

The blog’s name, “Fiery Spinning Sword”, is a biblical reference. It is a possible interpretation of “lahat hacherev hamit’hapechet” (Genesis 3:24, להט החרב המתהפכת) – an enchanted sword placed in the eastern entrance to the garden of Eden as a guardian of the tree of life; hovering in the air, spinning around and shredding any would-be trespasser wishing to eat its fruit. It is also a homage to the screen name “Holy-Fire” I use occasionally (which is not, incidentally, a religious reference), and to my passionate desire to cut through all the nonsense that you hear from time to time.

I hope you enjoy it.

Meni