How many hardware engineers does it take to develop an artificial general intelligence?

None. It’s a software problem.

At a recent wrap-up party of Harry Potter and the Methods of Rationality I attended on Pi day, I overheard a discussion about friendly artificial intelligence. I think several errors were made in that discussion, but unfortunately the suboptimal acoustic situation at the venue prevented me from offering my two satoshis. But I figured this would make for an interesting blog post, since if one person makes these mistakes, there must be more than one. (Incidentally, what HP:MoR, FAI and Bitcoin all have in common, is that I’ve heard about them from LessWrong.)

So far, humanity has had great success at building artificial specific intelligences. These are machines that can perform well in specific tasks which were once doable only with human intelligence. We have calculators that operate faster and more accurately than any human. We have chess programs that can easily beat the strongest human players. We even have cars that drive themselves more safely than humans.

What we don’t have is an artificial general intelligence (AGI) – a machine that has our ability to adapt to a very wide range of circumstances and solve practical problems in diverse fields. What will it take to create such a thing?

An argument I’ve heard says, that at our current technology level, we can build a machine with some specific level of intelligence (using, say, a generic state-of-the-art machine learning algorithm, such as a neural network). With hardware advances and Moore’s law, we will be able to build smarter machines, until one day, a computer will be as intelligent as a human. Past that point, it was said, computers with better hardware will become even smarter than humans, and gradually widen the gap.

Mankind has always been fascinated by the ability of birds to fly, and dreamed of gaining this ability itself. And people tried to proactively pursue this dream… By building feathered contraptions that resembled bird wings, attaching them to their bodies, and waving their arms vigorously.

That didn’t work.

People didn’t succeed in flying by building up muscle strength and flapping their arms more forcefully. They did it by understanding how flight works – the laws of physics, and aerodynamics in particular – and using this understanding to design a machine that can fly given our requirements and the tools available to us. These machines, of course, have only a superficial resemblance to birds.

Taking an algorithm which is crudely inspired by how brains are supposedly built, running it on increasingly faster hardware, and hoping that eventually general intelligence will emerge, is also not going to work. Instead, we need to understand how intelligence works, and use that to write software that will elicit intelligence from the technical capabilities of our computing hardware. Given the reliability and sheer processing power of modern digital computers, it is likely we will end up with a machine which is more intelligent than a human.

What’s next? The machine won’t wait around for Moore’s law to double its processing power and give it an edge in intelligence. Rather, it will use its superior intelligence to modify its own source code and create a better intelligence than we mere humans could create. The result will be even smarter and create an even better AI, and so on. The whole thing can explode rather quickly into absurd levels of intelligence.

What will this absurdly intelligent machine do? Another argument I’ve heard is that, since we wrote the code, it will only do what we told it.

It is a fundamental fact of theoretical computer science that, given an arbitrary program, there is no general way to tell if running this program will go on forever or stop at some point. Knowing whether a program stops or not is a pretty basic thing, so this already demonstrates the absurdity in thinking that knowing the code means knowing what the code does.

But we don’t need to go as far as these abstractions. Chess playing software were written by people, and these people have a good idea of the general way the program will go about finding the best moves. What they don’t know is what actual moves the program will play on the board. Indeed, chess programs often make moves no human would ever think of, because no human can do the trillions of calculations that the computer does.

But chess programs are just a specific intelligence. Once we build a program with general intelligence, we have no idea what specific course of action it will take. At first, we’ll have an idea about what the program does to reach a decision – but once the machine runs modified source code that it has written itself, we don’t even have that anymore.

It is generally assumed the AGI will be an “agent” – it will have a “target function”, a goal it wishes to achieve, and the software will be designed so that it always chooses the actions that best work toward this goal. We can try to construct the goal to be compatible with what we want, but “what we want” is incredibly complex and difficult to code; and the machine only cares about the goal we’ve written, not what we intended to write.

When we humans work towards our goals, we see fellow humans as our peers. When an AGI sees a human, it is more likely to see them as a collection of atoms that might be of more use to it in a different configuration. Avoiding the situation of a strong AI trampling humanity in pursuit of a naive target function that was coded into it, is exactly what the challenge of developing a Friendly artificial intelligence (FAI) is all about.

I’ve skipped over many details in this description, of course. But if you’re interested in learning more, you should stop listening to me – as I know nothing about this subject – and head over to https://intelligence.org/ (and if you ever decide to make a donation, they also accept Bitcoin).

What is real? And what is virtual?

The word “virtual” has several meanings. But the most obvious meaning people think of is “not real”. The uninformed often call Bitcoin a “virtual currency”, and contrast it with “real currency” such as the US dollar.

This is of course nonsensical. You could make stronger arguments for Bitcoin being real than for the US dollar. This is why Bitcoin should be more correctly referred to as a “digital currency”, emphasizing the fact that its existence consists in bits of digital information; or as a “cryptographic currency”, emphasizing that its operation is based on cryptography.

Similarly, the defining feature of the US dollar and its kin is that their issuing and usage is mandated by governments (regardless of whether an external body such as the Federal Reserve is charged with doing the actual issuing), and thus should be referred to as “government currency”. Alternatively, the terms “traditional currency” and “legacy currency” are slightly broader and emphasize that this is the kind of currency we have used so far. The term “fiat currency” is sometimes used to mean government currency, but personally I am not fond of that, as the literal meaning of the term is too broad.

A deeper understanding of the phrase “virtual currency” can be achieved only by contrasting it with something which is truly a virtual currency, such as World of Warcraft Gold. In the WoW game, virtual warriors are paying virtual gold to buy virtual swords with which to slay virtual dragons (or something like that. I’ve never actually played WoW).

None of this is real. Dragons of the kind featured in WoW have never existed in our physical universe. The WoW Gold does not correspond to any actual Au atoms. These are things that exist only in the virtual, simulated world.

The virtual dragons are encoded as bits, manifested as electrical and magnetic signals on computing devices somewhere. The bits are very real – they are configurations of actual electrons in our universe. But they are bits, not dragons.

Within the virtual world, the WoW gold is the ubiquitous currency used by the population, it is actual metal the people go out and mine, and it is not controlled by anyone. As such, the WoW gold is a virtual, physical, decentralized currency.

But just as the virtual dragons double as real bits, the WoW gold doubles as a real, digital, centralized currency.

The owners of this currency are the real players with user accounts that possess WoW Gold. And it has real value – real people offer real money (such as USD) to get WoW Gold, because they prefer to spend their real time playing WoW and slaying virtual dragons than farming virtual gold. And there are real sweatshops in China where real people work in real terrible conditions to play the game, earn WoW Gold, and sell it to aforementioned real players.

Of course, as a real digital currency, WoW Gold is centralized and is thus barely usable. It is completely controlled by Blizzard, hence is inefficient, and tricky to use because, AFAIK, its exchange for things of real value is an EULA violation. So WoW Gold is a bad real currency, whose saving grace is the extra semantics placed on it by the controlling company – which is entitlement to in-game virtual currency.

Since WoW Gold and its kin double as both digital currency and virtual currency, it is easy to see why people would get confused. But the contrast with Bitcoin becomes clear: It is digital, but it is not virtual. There is no virtual world in which Bitcoin is the currency. Rather, it is a currency used in our real world to pay for real products and services. It is digital and still fledgling, to be sure – but unlike WoW Gold, which is centralized and thus a bad currency, Bitcoin is decentralized and has all it takes to become a ubiquitous currency.

When in doubt, we should remember – that which vanishes is virtual, that which remains is real.

My new PC

It has been a while since I last posted on this blog; and the more this goes on, the more I feel that the post to break this prolonged absence must be “special”, and hence, harder for me to write. I’ve decided to break the cycle by writing a post that is special in the sense that it deviates from my usual habit of offering my thoughts on various developments in the Bitcoin world (which, incidentally, has celebrated its 6th birthday this week), and instead share my experiences from an important life event. Don’t worry, the word “Bitcoin” will still be mentioned several times.

I am talking, of course, about the new computer I have purchased. I’ll give you a moment to mock how uneventful my life must be if the purchase of a new computer is important, and move on to describe some of my design considerations.

Continue reading

The Creators and the Destructors

Some men and women are creators. They believe in dedicating their time and wealth to advancing a cause. They build things for others to enjoy, they help, they respect those around them in words and actions. They put their personal interests behind the greater good, and they always do the right thing, or at least try to. Over years of effort they prove their integrity and wisdom. If they do not have the means to contribute, they offer their gratitude and support to those who do.

When they disagree, they talk, ask questions, discuss. They make an honest effort to reach a mutual understanding in a respectful way. If when everything is clear the disagreement persists, they offer constructive criticism, they cherish the good and work to fix the bad.

But some have no interest in creation, only in destruction. They make little contribution of their own and impose impossible and arbitrary standards on those that do. They attack and slander the creators without correlation to what they do right or wrong. Whatever they have a gripe about, they strive to tear it down completely. When they disagree, they condone hateful speech, bullying, verbal abuse and threats of physical violence. Their only way of making themselves heard is by insulting those in front of them.

Bitcoin is only as strong as its community. When the goings get tough, when the community is torn asunder with infighting, when good men are subject to humiliation, when there is no unity, camaraderie or love, I take comfort in reminding myself who I’m struggling for. I’m not here for the destructors; I’m here for the creators.

Multi-PPS: Dissolving mining pool centralization

The Bitcoinsphere is abuzz with news of the GHash.io mining pool exceeding 50% of the Bitcoin network hashrate, which is of course in contrast with Bitcoin’s decentralized vision.

While it is indeed worrisome, I personally believe it is not something to panic about, as the underlying causes are solvable and we should not see such a phenomenon recur going forward.

The current centralization is the result of two distinct causes. One is the concentration of mining within a few large mining farms, often controlled by those who manufacture mining ASIC chips. These either take the mining risk themselves or forward it by offering mining contracts to the public. This is made possible by the economies of scale in building machines and operating them that large entities enjoy. This is indeed a problem, however, I believe this will be alleviated somewhat as SHA-256 computation becomes more of a commodity, as I claimed in ASIC will not Centralize Bitcoin Mining.

The second cause is that even individual miners who operate their own machine independently, use these machines in conjunction with large mining pools. The reason is, that with all classical mining pool reward methods, the larger a pool is, the better performance it can offer its miners in terms of the tradeoff between variance, fees and maturity time. A large pool will be more lucrative and hence attract more miners and become even larger, with the ultimate limit being consolidation within one pool.

Fortunately, I believe the latter cause is easily solvable. All it takes is using a reward framework that does not lead to “the rich get richer” effect. P2pool is often cited as a possible such framework, but while it has its merits, I do not believe it has what it takes to offer a sustainable solution, mostly due to the difficulty of small miners to use it.

Starting with p2pool as a basis, thinking what can be done with it and following it to its logical conclusion, I arrived at Multi-PPS, which I wrote about a year ago. Its basic premise is that miners should mine in multiple pools simultaneously, in proportion to each pool’s strength. Using this framework has two important features:

  1. The miner enjoys performance that is equivalent to that of a pool with a combined size of all pools he uses together.
  2. The stable equilibrium is not consolidation in one pool, but rather, maintaining a distribution between many pools according to the merits of each.

Of course, implementing this is nontrivial, and there are some technical performance issues to consider; however, the numbers do look promising, and the effort in implementing it is small relative to the potential gain. More details are available in the linked post.

Because this is such a hotly debated topic now and this proposed solution receives too little attention for my taste, I have decided to choose it as the topic for my technical talk in the upcoming Inside Bitcoin Tel Aviv Conference at the end of July.

Mtgox post mortem

(I’m assuming the rumors of Mtgox’s demise are not exaggerated, though it’s too early to tell).

It can currently be expected that whoever had money stored in Mtgox, will not see it again.

I am truly sorry for all those who have lost their money this way, certainly if this is a significant amount or even the entirety of their bitcoins.

But I also believe that people should take responsibility for their decisions and not expect too much sympathy if they make mistakes that cause them harm. I, too, have done nonsensical things which cost me a lot of money, and I try not to blame for them anything or anyone but myself.

Holding a significant amount of bitcoins on Mtgox was a mistake. Not just any mistake, but an antithesis to a core idea of Bitcoin. Bitcoin is about being decentralized and not having to trust anyone else to use. So what is the point with trusting all of your money with someone else?

There is a simple rule for trust – weigh what the other side has to gain from embezzlement, and what he has to lose. Anyone holding quantities of money for many people has a very strong incentive to get up and run, even if he has a successful business. 4 years in the market is really not enough time to build a reputation and trust that will justify deposits in the amounts Mtgox had.

And it’s not like Mtgox had a clear track record in these years. They’ve had lots of hacks and a variety of problems, and for almost a year it’s been nearly impossible to withdraw USD and there have been talks about insolvency.

Perhaps in a few years there will be a service reliable enough for the simple individual to trust with their money (preferably with a multisig arrangement or suchlike). But for now we are all early adopters and we should make the effort to learn how to store our bitcoins securely. It’s not trivial but not rocket science, and there are guides such as my previous post about paper wallets. Even with some basic methods you can reach a high level of surety. We hear all the time about people losing money they trusted with someone else, but despite all the talk of hackers we rarely hear of people whose own wallets were hacked. One can also divide his funds between different places (one of which could have been Mtgox) so that if anything happens one would not remain with nothing.

Back to the subject at hand. There are two problems with holding bitcoins on Mtgox. One is that an Mtgox wallet is not really designed for everyday use, so whoever has his money there is in fact stating that he has no interest in using Bitcoin but sees it purely as a speculative investment. This is not a problem in itself – it is legitimate to believe in Bitcoin and to want to profit from this belief. But part of this belief should be a belief in using Bitcoin in the way it was designed. And putting all of your bitcoins with someone without sufficient reputation and without layers of cryptographic or regulatory protection is not the way Bitcoin was meant to be used. So in essence, whoever has put his money with Mtgox is looking to profit from a belief but is not really acting on this belief. (Either that, or he doesn’t really believe in Bitcoin and simply wants to ride a trend, which is worse.)

For a long time people proficient with Bitcoin have been recommending not to hold funds in shared wallets such as Mtgox. I point that out not for gloating or a childish “told you so”, but simply to clarify that this is not hindsight, putting money in Mtgox is problematic and one could have known it was problematic. This must be taken in consideration to understand that the problem is not with Bitcoin, and to learn from it how to be more careful in the future.

I should clarify that exchanges are necessary, but this is what they are, exchanges. People who want to invest in Bitcoin, should buy and get them out of the exchange. People who want to day trade, should weight the profits of trading against the risks of holding money there.

The summary of the Mtgox issue is that (probably) an exchange that has been struggling for a long time is shutting down, and people who acted against the spirit of Bitcoin lost from it. The implications for Bitcoin are not major, beyond a self-fulfilling prophecy – people think this has implications for Bitcoin, and that itself harms Bitcoin.

I hope we all learn something from this story, and that despite the pain we will not stop believing in Bitcoin. It is not the currency which has failed us, it is us who have failed the currency.

How to create a reasonably secure Bitcoin paper wallet

tl;dr: If you don’t want the explanations, just follow the bolded instructions.

Some of us want to store a sizable amount of bitcoins as a long-term investment or savings. We need a way to store our keys (the pieces of data that give us, and only us, access to our coins) that will be safe from both loss and theft; but we do not care about them being easily usable on a daily basis. A great way to do this is paper wallets – storing our keys on printed pieces of paper.

Following this tutorial will allow you to create your very own paper wallet.

1. Go to https://raw.github.com/pointbiz/bitaddress.org/master/bitaddress.org.html. This is essentially a program that allows you to create keys (and corresponding Bitcoin addresses) in an easily printable form. Ignore the text you see when visiting the page.

2. In your browser’s menu, choose “Save page as” to save this file somewhere on your computer. All browsers have an option to do this, even if differently named. if you can’t find it try simply pressing Ctrl+S.

3. Disconnect your computer from the internet. This is very important. Your secure, offline paper wallet will be of no use if the generated keys are sent to an attacker by malicious software through an internet connection.

4. Open the file bitaddress.org.html you saved earlier (by double-clicking it). You’re not going to get your keys from the website’s server – rather, the page you’ve downloaded contains the program needed to generate the keys locally on your computer, using your browser. For this, it is best to download a copy of the page. Preferably, you should disable any addons running on your browser, for example, by running in incognito mode.

5. Go to the “Bulk Wallet” tab. The site contains many different options for generating paper wallets, but I find this one to be the best. It’s no-nonsense and allows you to print many addresses at once.

6. Choose the number of addresses you wish to print (e.g., 50), fill it in “Rows to generate”, and click “Generate”. Use as many as you can fit in a printed page in big enough letters to be easily readable.

7. Connect a printer with a USB cable. Of course you will need a printer for this. It’s safest to disconnect the computer entirely from any network, so this leaves USB for the connection. (I’m assuming you’re not using a parallel port for this…)

8. Click the page’s “Print” button to print several copies of a paper with the keys and corresponding addresses.

9. Create a file with the Bitcoin addresses (the shorter strings starting with “1”).

10. Make sure you have not saved the private keys (the longer strings starting with “5”, “K” or “L”) anywhere on the computer.

11. Clear your browser cache. This will decrease the chance that your browser keeps the keys. Instructions on how to do this on various browsers is available at http://www.wikihow.com/Clear-Your-Browser%27s-Cache.

12. Reset the computer. This will prevent malware from sending keys it has stored in the computer’s memory.

13. Reconnect the computer to the internet. It is safe to do so now with any trace of the keys erased.

14. Store the paper copies in multiple secure locations – a safe in your house, a relative, a safety deposit box, etc. Keep in mind that if a thief that understands Bitcoin gets his hand on one of the copies, the coins are gone. If all of the papers are lost or damaged to the point of illegibility, the coins are gone.

15. Send the bitcoins you’re saving to addresses in your paper wallet. You could send all of them to one address, but there are several advantages not to store too much on a single address. Split the coins however you see fit. You can always add more coins to any of the addresses. You can save most of the addresses you printed for future use. Keep in mind – once you send from an address, it is no longer secure and you will need to store your funds in the other addresses.

16. Eventually, you’ll want to actually use the funds stored on the paper wallet. That’s a problem for future you, but it is still useful to try this out so you know you can. Most clients have a feature for “importing” a private key – including Bitcoin-QT, blockchain.info and some hosted wallets. Try this option in your client of choice. The most secure method is to import the key into an offline wallet – for example, using Armory. This is fairly complicated now, but hopefully by the time you want to do it, the software and hardware to make it easier will exist.

Extra credit: As the title implies, this procedure will allow you to create a reasonably secure paper wallet. I’ve avoided some cautionary measures that I deemed would deter most users – perfect is the enemy of the good. If you do, however, want a more secure wallet, here are some more things you could do:

1. Make sure bitaddress.org is what it should be. In theory, the website from which you download the generation software could be hacked and replaced with malicious content. A simple way to protect against this is to search the web for mentions of “bitaddress was hacked”. A harder one is to compare the webpage’s SHA1 hash against the digitally signed hash.

2. Use a Linux live CD. Linux is less vulnerable to malware than Windows. This is true regardless of whether the reason is Linux being inherently more secure or hackers not bothering writing malware for it. If you don’t already use Linux, you don’t even have to install it – you can run it directly from the installation disc. For example, you can use Ubuntu for this – burn an Ubuntu live CD, and follow the procedure after booting from it.

3. Use a multi-factor system, such as multisig, SSSS or key encryption. You can encrypt your keys before printing them, so that stealing the paper will be useless without the password. However, a password strong enough to offer meaningful protection will be too hard to remember – you’d probably want to write it down; so this is really a primitive version of SSSS. Shamir’s Secret Sharing Scheme allows you to split the keys into n pieces, so that m pieces are required to recover the key. You can place the pieces in different places knowing that several pieces will need to be stolen to compromise the coins. The security of this, however, still relies on the computer in which the key is generated and split and on which it is combined and used. Using a multi-signature address is more secure, but also more complicated.

4. Don’t trust your printer – printers have memory which, combined with malicious firmware, could send their copies to an attacker once connected to the network. I don’t think this is a very serious risk, and I don’t know of countermeasures other than forgoing the printer and hand-writing the keys.

Additional tips:

1. Not all fonts are created equal. In some it is difficult to tell capital letters from small ones, which may create problems when you try to import the keys. Choose a clear font.

2. Consider laminating the papers, or using similar measures to protect them from damage.

3. bitaddress.org also allows creating QR codes for the keys. This is more space-consuming, but allows easier import.

Disclaimer: Use at your own risk! Even with the best security practices there can be failures; and these suggestions aim to balance security with accessibility rather than maximize security. I offer absolutely no warranty or guarantee and will take absolutely no responsibility if this guide leads, either directly or indirectly, to loss of coins or any other harm.