How the blockchain actually works

2024 Author: Seth Attwood | [email protected]. Last modified: 2023-12-16 15:55

This post is intended to tell everyone why the blockchain was invented, how cryptocurrencies work and why it is the most beautiful system in recent years from the point of view of logic.

I will immediately warn you that under the cut there is a HUGE sheet of text and if you are not ready to "close" the question on the topic of cryptocurrencies once and for all, add an entry to your favorites right now and reserve time)

Blockchain is a technology, new, strange, incomprehensible, but it seems to be changing the world, in contrast to these stories of yours. Apparently, she is with us for a long time.

This post is written as if it is being told to people who are very far from computers and know them only superficially, for example, imagine that it was preparing for your parents. I can even throw it off to my fellow humanities and be sure they will understand.

And Oleg will help us all in this difficult matter. Meet!

If you don't like Oleg, I'll kick him out.

The basics: why do we need blockchain?

The blockchain was described in Satoshi Nakamoto's article "Bitcoin: A Peer-to-Peer Electronic Cash System". There, in just eight pages, the author described the basics of the Bitcoin cryptocurrency, which was based on the Blockchain algorithm.

List that cannot be changed

Blockchain - a chain of blocks or in other words a linked list. In such a list, each next record refers to one previous one, and so on down the chain to the very first. Like the carriages of a train, each one drags the next one along. Regarding the lists, there is a good article by Nikita Likhachev on TJ, where the same is explained for completely newbies. The analogies are partly taken from there.

Let's take an example

Oleg's friends constantly borrow money from him. Oleg is kind, but extremely forgetful. A week later, he no longer remembers who did not return the debt to him, but he is embarrassed to ask everyone about it. Therefore, one day he decided to put an end to this by creating a list of friends on the blackboard at his home to whom he borrowed money.

Now Oleg can always go to the board and make sure that Max returned everything, but Vanya does not give 700 rubles already. One day Oleg invites Vanya to have a drink at his home. While Oleg is going to the toilet, Vanya erases the entry “I borrowed 200 rubles for Vanya” and writes in its place “Vanya gave 500 rubles”.

Oleg, who trusted his list, forgets about the debt and loses 700 rubles. He decides to somehow fight it. Last year, Oleg attended a programming course, where he was told about hashing. He remembers that any string can be turned into an unambiguous set of characters - a hash, and changing any character in the string will completely change it.

Adding a dot at the end changed the final hash beyond recognition - you can use it.

Oleg takes the well-known SHA-256 hash and hashes each record with it, adding the result at the end. Now Oleg can make sure that no one has changed his records by hashing them again and comparing them with the green one.

But EVIL IVAN also knows how to use SHA-256 and can easily change the entry along with its hash. Especially if the hash is written right next to it on the board.

Therefore, for greater security, Oleg decides to hash not only the record itself, but add it together with the hash from the previous record. Now all the following entries depend on the previous ones. If you change at least one line, you will have to recalculate the hashes of all the others below in the list.

But one day Ivan sneaks up at night, changes the entry he needs and updates the hashes for the entire list to the end. It takes him several hours, but Oleg is still fast asleep and cannot hear. In the morning, Oleg discovers an absolutely correct list - all hashes match. But Ivan deceived him anyway, even though he spent a sleepless night on it. How else can you protect yourself from Night Ivan?

Oleg decides to somehow complicate his life. Now, to add a new entry to the list, Oleg will solve a complex problem associated with it, for example, a mathematical equation. He will add the answer to the final hash.

Oleg is good at mathematics, but even it takes ten minutes to add an entry. Despite this, the time spent is worth it, because if Ivan wants to change something again, he will have to re-solve the equations for each row, and there can be dozens of them. It will take a lot of time, because the equations are unique each time and associated with a specific record.

But checking the list is just as simple: first you need to compare the hashes as before, and then check the solutions of the equations by simple substitution. If everything converges, the list is not changed.

In reality, things are not so good with equations: computers solve them too well, and where to store so many unique equations. Therefore, the authors of the blockchain came up with a more beautiful problem: you need to find such a number (nonce) so that the final hash of the entire record starts with 10 zeros. Such a nonce is difficult to find, but the result can always be checked with just eyes.

Now Oleg verifies all the hashes and additionally makes sure that each begins with the specified number of zeros. Sly Ivan, even armed with a powerful laptop, will not have time to recalculate all the hashes in a night so that they satisfy the condition - there will not be enough time.

Such a list, in fact, is the home blockchain on its knee. Its safety is guaranteed by mathematicians, who proved that these hashes cannot be calculated somehow faster, except by brute force. Such enumeration of hashes for each record is mining, about which today there will be a lot and in detail.

Centralization of trust

Our friends liked the idea of keeping a fake list of "who borrowed whom". They also don’t want to remember who paid for whom in the bar and how much they still owes - everything is written on the wall. You discussed the idea and decided that now you need a single list for all.

But who should be entrusted with such important bookkeeping? After all, when it comes to money, trust comes to the fore. We will not trust the unknown to keep our money. For this, our ancestors invented banks, which over time began to be trusted, because they are backed by a license, laws and insurance from the Central Bank.

In the circle of friends, everyone trusts each other and you can simply choose the most responsible one for this role. But what if the question is about strangers? An entire city, country, or the whole world, as is the case with Bitcoin? In general, no one can trust anyone there.

Decentralization: no one trusts anyone

So they came up with an alternative approach: keep a copy of the list for everyone. Thus, an attacker would not only have to rewrite one list, but also sneak into each house and rewrite the lists there. And then it turns out that someone kept several lists at home, which no one knew about. This is decentralization.

The downside of this approach is that in order to make new entries, you will have to call all other participants and inform each of them of the latest changes. But if these participants are soulless machines, it ceases to be any problem at all.

In such a system, there is no single point of trust, and hence the possibility of bribery and cheating. All participants in the system act according to a single rule: no one trusts anyone. Everyone believes only the information that they have at their disposal. This is the main law of any decentralized network.

Transactions

When buying a pot in a store, you enter the pin code from your card, allowing the store to ask the bank if you have 35 rubles on your account. In other words, you sign a transaction for 35 rubles with your pin code, which the bank confirms or rejects.

Our records of the type "I borrowed Vanya 500 rubles" are also transactions. But we do not have a bank authorizing the author of the transactions. How can we check that Ivan has not quietly added the entry "Max owes Oleg 100,500 rubles"?

The blockchain uses the mechanism of public and private keys for this; IT specialists have long been using them for authorization in the same SSH. I explained it on my fingers in the post "Security, encryption, cyberpunk" in the "Introduction to encryption" section.

Briefly about how this complex but beautiful mathematics works: you generate a pair of long prime numbers on your computer - a public and a private key. The private key is considered super secret because it can decrypt what is encrypted in the public.

But the opposite also works. If you share the public key with all your friends, they will be able to encrypt any message for them so that only you can read it, since you own the private one.

But besides this, the public key has a useful effect - with it you can check that the data was encrypted with your private key, without decrypting the data itself. All these properties are well described in the "Book of Ciphers".

We are on a decentralized internet where no one can be trusted. The transaction is signed with a private key and, together with the public key, is sent to a special storage - a pool of unconfirmed transactions. So any member of the network can verify that it was you who initiated it, and not someone else wants to pay off with your money.

This ensures the openness and security of the network. If earlier banks were responsible for this, then in the blockchain, mathematicians are responsible for this.

For ordinary users who do not want to figure out how to issue and store private keys, online wallet services will help. To copy long public keys, convenient QR codes are made there. For example Blockchain Wallet, because it has a convenient mobile application and supports the two main cryptocurrencies - BTC and ETH.

Lack of the concept of "balance"

Like our board, the blockchain essentially consists of only transaction history. It does not store the balance of each wallet, otherwise we would have to invent additional protection methods.

Ownership of the wallet is confirmed only by the private key. But how can other members of the network make sure I have enough money to buy?

Since we have no balance, you must prove it. Therefore, a blockchain transaction includes not only your signature and how much you want to spend, but also links to previous transactions in which you received the required amount of money.

That is, if you want to spend 400 rubles, you run through your entire history of income and expenses, and attach to your transaction those incomes where you were given 100 + 250 + 50 rubles, thereby proving that you have these 400 rubles.

Each member of the network will once again be sure to check that you have not attached income twice. That those 300 rubles that Max gave last week, you really haven't spent yet.

Such incomes attached to a transaction are called inputs in the blockchain, and all recipients of money are called outputs. The sum of all inputs is rarely exactly the same as you want to transfer at a time - therefore, one of the outputs will most often be yourself. In other words, the transaction on the blockchain looks like "I was given 3 and 2 BTC, I want to transfer 4 BTC from them and return the remaining 1 BTC back."

The beauty of blockchain is that inputs don't have to come from one wallet. After all, only the key is checked. If you know the private key of all inputs, then you can easily attach them to your transaction and pay with this money. As if you were paying in a supermarket with several cards from which you know the pin code.

However, if you lose your private key, your disk dies or your laptop is stolen, your bitcoins will be lost forever. Nobody can use them as inputs for new transactions.

This amount will be inaccessible to the whole world forever - as if you burn a bundle of banknotes. There is no single bank where you can write an application with a copy of your passport, and he will print it. This requires an additional release of new bitcoins "out of thin air".

Double spending problem

Above I said that transactions are added to a special "pool of unconfirmed transactions". Why do we need some kind of intermediate entity, if we already have, in fact, ready-made signed transactions? Why not write them directly to the blockchain?

Because the signals from point A to point B always go with a delay. Two transactions can go completely different ways. And the transaction that was initiated first can reach the recipient later, because it took a longer path.

This results in double spending, when the same money was sent to two addressees at once, which they would not even guess about. This is not to hand over bills from hand to hand.

For a decentralized network in which no one can be trusted, this problem is especially acute. Here's how you make sure that one transaction was definitely before another? Ask the sender to sew the dispatch time into it, right? But remember - you can't trust anyone, not even the sender.

The time on all computers will necessarily differ and there is no guaranteed way to synchronize them. A copy of the blockchain is stored on every computer in the network and each participant trusts only it.

How can you make sure that one transaction was earlier than another?

The answer is simple: it’s impossible. There is no way to confirm the time of a transaction on a decentralized network. And the solution to this problem is the third important blockchain idea that Satoshi invented and which, oddly enough, is spelled out right in his name - blocks.

Blocks are the backbone of the blockchain

Each working computer on the network selects any transactions that it likes from the general pool. Usually just for the highest commission he can earn on it. So he collects transactions for himself until their total size reaches the negotiated limit. In Bitcoin, this block size limit is 1 MB (after SegWit2x it will be 2 MB), and in Bitcoin Cash - 8 MB.

The whole blockchain is essentially a list of such blocks, where each one refers to the previous one. It can be used to track any transaction in the entire history, unwinding the blockchain even to the very first record.

It is this list that now weighs hundreds of gigabytes and must be fully downloaded to all computers that want to take part in the network (but this is not necessary to simply create transactions and transfer money). It is downloaded the same way from all the nearest computers on the network, as if you are downloading a series from torrents, only new new episodes are released in it every 10 minutes.

Having typed transactions from the pool for itself, the computer begins to compose from them the same unforgeable list as we did at the beginning of the post on the board at home.

Only he makes it in the form of a tree - hashes the records in pairs, then the result is again in pairs, and so on until there is only one hash left - the root of the tree, which is added to the block. Why exactly with a tree - I did not find the answer, but I suppose that it is simply faster that way. Read more on the wiki: Merkle tree.

Since the current blockchain has already been downloaded, our computer knows exactly what the last block is in it. He just needs to add a link to it in the block header, hash it all and tell all other computers on the network “look, I made a new block, let's add it to our blockchain”.

The rest should check that the block is built according to all the rules and that we have not added unnecessary transactions there, and then add them to our chains. Now all transactions in it are confirmed, the blockchain is increased by one block and everything is going well, isn't it?

But no. Thousands of computers are simultaneously working on the network, and as soon as they assemble a new block, they almost simultaneously rush to inform everyone that their block was created first. And from the previous section, we already know that in a decentralized network it is impossible to prove who really was the first.

As in school, when everyone was solving a difficult test, it rarely happened that even excellent students took the answers at the same time.

But if for a person it is a difficult task to plan a vacation so that he gets on the May holidays, and tickets to the sea are inexpensive, then for a computer it is to add such a number (nonce) to the end of the block so that as a result, the SHA-256 hash for the entire block begins let's say 10 zeros. This is the problem that needs to be solved in order to add a block to the Bitcoin network. For other networks, the tasks may differ.

So we come to the concept of mining, which everyone has been so obsessed with in recent years.

Mining

Bitcoin mining is not some kind of sacred sacrament. Mining is not about finding new bitcoins somewhere in the depths of the Internet. Mining is when thousands of computers around the world are buzzing in basements, going through millions of numbers per second, trying to find a hash that starts with 10 zeros. They don't even need to be online to do this.

Video cards with hundreds of parallel cores solve this problem faster than any CPU.

Why exactly 10 zeros? And just like that, it makes no sense. So Satoshi came up with. Because this is one of those problems for which there is always a solution, but it certainly cannot be found faster than a long monotonous enumeration of options.

The complexity of mining directly depends on the size of the network, that is, its total power. If you create your own blockchain and run it yourself at home on two laptops, then the task should be simpler. For example, so that the hash starts with only one zero, or so that the sum of the even digits is equal to the sum of the odd ones.

It will take several decades for one computer to find a hash starting with 10 zeros. But if you combine thousands of computers into a single network and search in parallel, then, according to the theory of probability, this problem is solved on average in 10 minutes. This is the time when a new block appears in the bitcoin blockchain.

Every 8-12 minutes, someone on earth finds such a hash and gets the privilege of announcing their discovery to everyone, thereby avoiding the problem of who was the first.

For finding an answer, the computer (as of 2017) receives 12.5 BTC - this is the amount of reward that is generated by the bitcoin system "out of thin air" and decreases every four years.

Technically, this means that each miner always adds another transaction to his block - “create 12.5 BTC and send them to my wallet”. When you hear "the number of bitcoins in the world is limited to 21 million, now they have already hired 16 million" - these are the rewards generated by the network.

Go see a real live Bitcoin block on one of the dedicated sites. There are also transactions with inputs and outputs, and as many as 18 zeros at the beginning and all the hashes described above.

It is the miners who add the emerging transactions to the blockchain. So if someone tells you that he will "make a blockchain for ***", the first question he must answer is who will mine on it and why. Most often, the correct answer is “everyone will, because for mining we give our coins, which will grow and it is profitable for miners”. But this does not apply to all projects.

For example, some Ministry of Health tomorrow creates its own closed blockchain for doctors (and they want it), who will mine it? Weekend therapists?

But what will be the benefit to miners later, when the rewards disappear or become scanty?

According to the Creator's idea, by that time people will have to believe in the reality of bitcoin and mining will begin to pay off by the amount of fees included in each transaction. This is where everything goes: back in 2012, all commissions were zero, miners mined only for rewards from blocks. Today, a transaction with zero commission can hang in the pool for several hours, because there is competition and people are willing to pay for speed.

That is, the essence of mining is in solving meaningless problems. Couldn't all this power be used for something more useful - the search for a cure for cancer, for example?

The essence of mining is to solve any computational problem. This task should be simple enough for network participants to have a stable probability of finding an answer - otherwise, transactions will be confirmed forever. Imagine that at the checkout in a store you have to wait half an hour each time for the bank to confirm your transaction. Nobody will use such a bank.

But the task must be at the same time difficult, so that all netizens do not find the answer at once. Because in this case they will announce many blocks with the same transactions to the network and there will be a possibility of "double waste", which I mentioned. Or even worse - dividing a single blockchain into several branches, in which no one can figure out which transaction is confirmed and which is not.

If a reward of 12.5 BTC is handed out only once every 10 minutes and only one person who found the block, it turns out that I have to waste video cards for several years in the hope that one day I will drop $ 40,000 (at the current exchange rate)?

This is exactly the case for bitcoin. But it was not always so. Previously, the network was smaller, the complexity is lower, which means that the probability of single-handedly finding a hash for a new block is higher. But then bitcoin was not so expensive.

Now no one is mining bitcoins alone. Now the participants are united in special groups - mining pools, where everyone together is trying to find the correct hash.

If at least one of the group finds, then the entire reward is divided between the participants, depending on their contribution to the common work. It turns out that you are mining and you get a penny from the total share every week.

But solo mining is quite possible on other networks. Until recently, it was easy to mine Ethereum, where blocks are found every 10 seconds. The block reward is much lower there, but the probability of earning a pretty penny is higher.

So we are going to burn thousands of video cards in vain and there is no way out?

Yes, but there are ideas. The mining that I described is classic and is called Proof-of-Work (proof of work). That is, each machine proves that it worked for the benefit of the network by solving meaningless problems with a given probability.

But some guys are starting to make blockchains with other types of mining. Now the second most popular concept is Proof-of-Stake (proof of stake). In this type of mining, the more “coins” a network participant has in his account, the more likely he is to insert his block into the blockchain. Like the loudest guy in the village.

You can think of other types of mining. As already suggested, all computers on the network can look for a cure for cancer, only you need to figure out how, in this case, to record their contribution to the system. After all, I can declare that I am also participating, but turn off my video card and count nothing.

How do you quantify each participant's contribution to finding a cure for cancer? If you come up with it - dare to cut your CancerCoin, a hype in the media is guaranteed to you.

Blockchain

Imagine a situation in which, despite all our theory of probability, two miners still managed to find the right answer at the same time. They start sending two absolutely correct blocks over the network.

These blocks are guaranteed to be different, because even if they miraculously chose the same transactions from the pool, made absolutely identical trees and guessed the same random number (nonce), their hashes will still be different, since each will write his wallet number in the block for reward.

Now we have two valid blocks and again the problem arises of who should be considered first. How will the network behave in this case?

The blockchain algorithm states that network participants simply accept the first correct answer that reaches them. Then they live on the basis of their own picture of the world.

Both miners will receive their reward, and all the rest begin to mine, relying on the last block they personally received, discarding all the rest are re-correct. Two versions of the correct blockchain appear on the network. Such is the paradox.

This is a routine situation in which probability theory helps again. The network functions in such a bifurcated state until one of the miners finds the next block to one of these chains.

As soon as such a block is found and inserted into the chain, it becomes longer and one of the agreements of the blockchain network is included: under any conditions, the longest blockchain is accepted as the only true one for the entire network.

The short chain, despite all its correctness, is rejected by all participants in the network. Transactions from it are returned to the pool (if they have not been confirmed in another), and their processing begins anew. The miner loses his reward because his block no longer exists.

With the growth of the network, such coincidences from "very unlikely" go into the category of "well, sometimes it happens." Old-timers say that there were cases when a chain of four blocks was dropped at once.

Because of this, three end of chain insecurity rules have been invented:

1. Rewards for mining can be used only after another 20 confirmed blocks after receiving. For Bitcoin, this is about three hours.

2. If bitcoins were sent to you, you can use them as inputs in new transactions only after 1-5 blocks.

3. Rules 1 and 2 are just spelled out in the settings of each client. Nobody monitors their observance. But the longest chain law will still wipe out all your transactions if you try to trick the system into not enforcing them.

Trying to cheat the blockchain

Now that you know everything about mining, the structure of the blockchain and the rule of the longest chain, you might have a question: is it possible to somehow specifically overtake the blockchain by making the longest chain yourself, thereby confirming your fake transactions.

Let's say you have the most powerful computer on earth. Google and Amazon datacenters put together at your disposal and you are trying to calculate such a chain that will become the longest blockchain in the network.

You cannot take and immediately calculate several blocks of the chain, because each next block depends on the previous one. Then you decide as quickly as possible to count each block on your huge datacenters in parallel with how all other participants continue to increase the main blockchain. Is it possible to overtake them? Probably yes.

If your computing power is more than 50% of the power of all network participants, then with a 50% probability you will be able to build a longer chain faster than all the others combined. This is a theoretically possible way to trick the blockchain by calculating a longer chain of transactions. Then all transactions of the real network will be considered invalid, and you will collect all the rewards and start a new milestone in the history of cryptocurrency, which is called "blockchain division". Once, due to a bug in the code, this was the case with Ethereum.

But in reality, no data center can be compared in terms of power to all computers in the world. One and a half billion Chinese with icics, another one and a half billion hungry Indians with mining farms and cheap electricity - this is huge computing power. No one in the world can yet compete with them alone, not even Google.

It's like going out on the street and trying to convince every person in the world that the dollar is now worth 1 ruble and be in time before the media exposes you. And if you manage to convince everyone, you can collapse the world economy. In theory, isn't it possible? But in practice, for some reason, no one succeeded.

The blockchain also rests on this probability. The more participants-miners, the more security and trust in the network. Therefore, when another large mining farm is covered in China, the rate collapses. Everyone is afraid that somewhere in the world there was an evil genius who had already collected a pool of miners at ~ 49% capacity.

Conclusion

Blockchain is not a strictly defined set of algorithms. It is a structure for building a fake network between participants, where no one can trust anyone. While reading, you probably had the thought more than once that “you can do it like this and it will be even more useful”. This means you understand the blockchain, congratulations.

Some guys in the world also understood it and wanted to improve or adapt for some specific tasks. Cryptocurrencies are not the same, although there are a lot of them too. Here is a short list of some of the ideas and projects that have gained some popularity thanks to the rethinking of the blockchain idea.

Ethereum

“Ethers” is the second most popular word you hear in crypto-hype news, after Bitcoin. For ordinary people, this is another cryptocurrency and a way to do the trendiest thing called ICO. The developers on the site describe Ethereum as "a blockchain builder for your needs." This is also possible, yes.

But if you dig even deeper, ether is not just a network of coins. This is a huge global computing machine, where users execute the code of other people's programs (smart contracts), receiving a reward for each line executed. And all this is decentralized, indestructible and with all the guarantees of the blockchain.

We can talk about Ethereum and smart contracts for so long that it will be enough for another such post. Therefore, we will act in the style of top bloggers: if this post is actively repost and retweeted, and by Friday it gets at least 1,500 unique views, I will write a sequel about Ethereum and smart contracts.