How does the Blockchain Work? (Part 1)

Via: Medium.com
Collin Thompson  Blockchain Product Designer & Growth Marketer

A simple and easy explanation

How does the Blockchain Work?

Well here is a simple explanation that cuts through the hype.

Blockchain is a hot topic around the world these days, yet for many, the technology remains an elusive concept. Yet it shouldn’t, the concept is simple once you get your head around the architecture and theory of basic crypto economics. When you do have your “a Ha” moment, the world will never seem the same to you again.

This blockchain basics guide is designed to deliver a clear, non-technical introduction to one of the most transformational & misunderstood technologies of our time. If you want to know what blockchain technology is, how it works, and it’s potential impacts, without all the technical lingo, then this post is for you.

A short History of Transacting Money

Historically, when it comes to transacting money or anything of value, people and businesses have relied heavily on intermediaries like banks and governments to ensure trust and certainty.[1] Middlemen perform a range of important tasks that help build trust into the transactional process like authentication & record keeping.[2]

The need for intermediaries is especially acute when making a digital transaction. Because digital assets like money, stocks & intellectual property, are essentially files, they are incredibly easy to reproduce. This creates what’s known as the double spending problem (the act of spending the same unit of value more than once) which until now has prevented the peer to peer transfer of digital assets.[3]

But what if there was a way of conducting digital transactions without a third party intermediary? Well, a new technology exists today that makes this possible. But before we dive into the mechanics of this revolutionary technology, it’s important to provide a little context.

Blockchain Vs Bitcoin — What’s the connection?

Bitcoin first appeared in a 2008 white paper authored by a person, or persons using the pseudonym Satoshi Nakamoto. The white paper detailed an innovative peer to peer electronic cash system called Bitcoin that enabled online payments to be transferred directly, without an intermediary.[4]

how the blockchain transfers value

Via (techliberation.com)

While the proposed bitcoin payment system was exciting and innovative, it was the mechanics of how it worked that was truly revolutionary. Shortly after the white paper’s release, it became evident that the main technical innovation was not the digital currency itself but the technology that lay behind it, known today as blockchain.

Although commonly associated with Bitcoin, blockchain technology has many other applications. Bitcoin is merely the first and most well-known uses. In fact, Bitcoin is only one of about seven hundred applications that use the blockchain operating system today.[5]

“[Blockchain] is to Bitcoin, what the internet is to email. A big electronic system, on top of which you can build applications. Currency is just one.” [6] — Sally Davies, FT Technology Reporter

One example of the evolution and broad application of blockchain, beyond digital currency, is the development of the Ethereum public blockchain, which is providing a way to execute peer to peer contracts.[7]

What’s under the blockchain hood?

Simply put, a blockchain is a type of distributed ledger or decentralized database that keeps continuously updated digital records of who owns what. Rather than having a central administrator like a traditional database, (think banks, governments & accountants), a distributed ledger has a network of replicated databases, synchronized via the internet and visible to anyone within the network.[8] Blockchain networks can be private with restricted membership similar to an intranet, or public, like the Internet, accessible to any person in the world.[9] [10]

When a digital transaction is carried out, it is grouped together in a cryptographically protected block with other transactions that have occurred in the last 10 minutes and sent out to the entire network. Miners (members in the network with high levels of computing power) then compete to validate the transactions by solving complex coded problems.[11] The first miner to solve the problems and validate the block receives a reward. (In the Bitcoin Blockchain network, for example, a miner would receive Bitcoins).

The validated block of transactions is then timestamped and added to a chain in a linear, chronological order. New blocks of validated transactions are linked to older blocks, making a chain of blocks that show every transaction made in the history of that blockchain.[12] The entire chain is continually updated so that every ledger in the network is the same, giving each member the ability to prove who owns what at any given time.

“A blockchain is a magic computer that anyone can upload programs to and leave the programs to self-execute, where the current and all previous states of every program are always publicly visible, and which carries a very strong crypto economically secured guarantee that programs running on the chain will continue to execute in exactly the way that the blockchain protocol specifies.” — Vitalik Buterin

Blockchain’s decentralized, open & cryptographic nature allow people to trust each other and transact peer to peer, making the need for intermediaries obsolete. This also brings unprecedented security benefits. Hacking attacks that commonly impact large centralized intermediaries like banks would be virtually impossible to pull off on the blockchain. For example — if someone wanted to hack into a particular block in a blockchain, a hacker would not only need to hack into that specific block, but all of the proceeding blocks going back the entire history of that blockchain. And they would need to do it on every ledger in the network, which could be millions, simultaneously.[13]

Will the blockchain transform the Internet & the global economy?

Make no mistake about it. Blockchain is a highly disruptive technology that promises to change the world as we know it. The technology is not only shifting the way we use the Internet, but it is also revolutionizing the global economy.[14]

By enabling the digitization of assets, blockchain is driving a fundamental shift from the Internet of information, where we can instantly view, exchange and communicate information to the Internet of value, where we can instantly exchange assets.[15] A new global economy of immediate value transfer is on its way, where big intermediaries no longer play a major role. An economy where trust is established not by central intermediaries but through consensus and complex computer code.[16]

“The technology likely to have the greatest impact on the next few decades has arrived. And it’s not social media. It’s not big data. It’s not robotics. It’s not even AI. You’ll be surprised to learn that it’s the underlying technology of digital currencies like Bitcoin. It’s called the blockchain.” — Don Tapscott

Blockchain has applications that go way beyond obvious things like digital currencies and money transfers. From electronic voting, smart contracts & digitally recorded property assets to patient health records management and proof of ownership for digital content.

Blockchain will profoundly disrupt hundreds of industries that rely on intermediaries, including banking, finance, academia, real estate, insurance, legal, health care and the public sector — amongst many others.[17] This will result in job losses and the complete transformation of entire industries. But overall, the elimination of intermediaries brings mostly positive benefits. Banks & governments for example, often impede the free flow of business because of the time it takes to process transactions and regulatory requirements. The blockchain will enable an increased amount of people and businesses to trade much more frequently and efficiently, significantly boosting local and international trade. Blockchain technology would also eliminate expensive intermediary fees that have become a burden on individuals and businesses, especially in the remittances space.

Perhaps most profoundly, blockchain promises to democratize & expand the global financial system. Giving people who have limited exposure to the global economy, better access to financial and payment systems and stronger protection against corruption and exploitation.

“Every human being on the planet with a phone, will have equal access. Expanding the total addressable market by 4X” — Brock Pierce

The potential impacts of blockchain technology on society and the global economy are hugely significant. With an ever growing list of real-world uses, blockchain technology promises to have a massive impact. This is just the beginning.

Many of the most exciting applications and platforms haven’t even been invented yet!

 

How does the Blockchain Work? (Part 2)

The top 5 things that you need to know.

Blockchain essentials for dummies
 Updated August 10, 2017.

The talk about Blockchain technology seems ubiquitous. But what exactly is a Blockchain? More specifically, what are the Blockchain essentials that you should really know?

Let’s dive in to find out more about and separate the hype from the reality

1. What is a Blockchain?

A Blockchain is a tamper-proof distributed public ledger that manages transactions.

Think of it like a magical Google spreadsheet in the cloud, or more specifically on a network.

Put simply, a Blockchain is basically an incorruptible distributed ledger of data, which can be used to store informational assets ranging from managing cryptographic contracts to transferring value.

The most recognized application on a blockchain is bitcoin transactions.

The transferring of value from one person to another with no central intermediary, and without allowing a person or party to spend their bitcoin twice “the double spend rule”.

What does this mean?

It means that “value” can have a change of title and ownership from one person/party to another, without the need of a trusted third party to validate/govern the trade.

How is that you might ask?

Well, the governance is in the protocol
(you will find more information on this below so keep reading)

Beside being a ledger for “data of value”, or cryptocurrencies, Blockchain technology is finding broader usage in peer to peer lending, (smart) contracts managements, healthcare data, stock transfers, and even elections.

Like any emerging and disruptive technology, no one can predict the future of Blockchain technology. But one thing’s for sure — it isn’t (just) for purchasing black-market goods and services!

As a matter of fact, Blockchain technology is finding its way into big firms such as IBM, Microsoft, and major banks.

Interest in the technology is driven by (fear of disruption) the fact that it excludes trusted third parties (banks and clearinghouses) during transfer of values, which in turn results in fast, private and less expensive financial transactions.

Blockchain can facilitate the peer-to-peer transfer of anything that’s of value.

This may range from assets, properties, and contracts. The most crucial and far-reaching Blockchain applications is applied in Bitcoin, with transfer of value, and Ethereum, with its enhancement of smart contracts.

Let’s jump in and learn the historical background of these Blockchain essentials.

2. Bitcoin

The Bitcoin currency, as many have come to know it, has been with us since 2008 when Satoshi Nakamoto — A person, or group of people, published a whitepaper about peer-to-peer electronic currency.

The major innovation that bitcoin unveiled was direct and secure transfer of money or “value” directly to any party on the network.

The Bitcoin currency network is decentralized — there’s no central authority — the underlying Blockchain technology is used to store information which is verified by a network of “miners” who validate all transactions on the network.

How should I think of this?

Bitcoin is simply a virtual currency system which resembles the real world cash system.

Since it’s un-eponymous launch in 2008, through the boom and bust of the hype cycle, Bitcoin has continued to grow at an exponential rate, and the fringe curiosity that consumed a group of highly capable (Tech Nerds) has ushered in some new upgrades that has brought blockchain closer to the mainstream.

3. Ethereum — Blockchain 2.0

Ethereum is a blockchain system based on the concepts of bitcoin.

It is considered a second generation blockchain technology that was designed to let any person, with a basic level of computer skills, to develop and deploy their own decentralized applications on the Blockchain.

Just like the Bitcoin, Ethereum is decentralized — no one regulates or owns it — it has it’s own cryptocurrency or “fuel” called “Ether” which acts in the same way bitcoin does. However, Ethereum has a few innovations worth noting. The first being a second application on its blockchain infrastructure called a “smart contract”, it’s own virtual machine which powers the memory and applications on the network called the “ethereum virtual Machine”, and it own programmable language called “Solidity”.

Ethereum is kinda like Bitcoin on steroids, but made to be more accessible.

It was developed by Vitalik Buterin, a 19 year old Russian Canadian in 2013 as a Blockchain 2.0 — next generation Blockchain technology — with capabilities to be able to program and perform, arbitrary and complex computations.

Rather than just providing users with a set of predefined operations — like Bitcoin transactions — Ethereum lets users develop their own operations with the complexity they wish.

4. Smart Contracts

What is a “smart” contract?
— Well they actually aren’t that “smart”

Think of them like self executing dumb software robots that live and do business on a decentralized network.

Smart contracts are autonomous computer systems, written in code, that manage executions between individuals on the Blockchain.

The code resides at specified addresses on the Ethereum Blockchain. These contracts are powered by our friend the Ethereum Virtual Machine (EVM) and by Ether. It’s the little engine that could, that keeps all the smart contracts running on time and coordinates them with the rest of the network.

In order to create an added layer of customization and security Ethereum created some high-level languages that are used to create smart contracts for the EVM. Solidity, Serpent, and LLL.

These are the major innovation that Ethereum has brought to blockchains and it allows for many amazing types of autonomous programs.

Next, let’s explore the consensus mechanisms in Blockchain.

5. Consensus Mechanisms

“When you interact with multiple parties, you need some sort of consensus mechanism to ensure everyone has got the right records”–Dan O’Prey, Co-founder of Hyperledger.

Both Bitcoin and Ethereum use a decentralized system to confirm the transactions without relying on a trusted third party.

Therefore, consensus, or coming to a uniform agreement, helps a network of autonomous programs and computers come to an agreed state of the blockchain without conflict.

As a matter of fact, the consensus is the backbone of the Blockchain and any other decentralized and distributed technology

The proof of work, proof of stake and closed consensus are the most common mechanisms used in Blockchain technologies.

A: Proof of work

The most common consensus mechanism that’s used for Blockchain technology is what’s called “proof of work”. It is the system used in Bitcoin.

When a transaction is initiated, the information is stored in a candidate block which is filled with the transaction’s information. A cryptographic beacon is sent out to the mining network that the candidate block has been created, and the miners get to work on solving a cryptographic puzzle that has a prize for whomever solves it, in the form of newly minted coins/currency.

Miners have what some would think of as supercomputers that are much more powerful than the average Person’s Macbook pro. These machines have a “hashrate” or computing power that gives them an advantage when competing to solve consensus problems for reward.

I know what all you climate control advocates are saying: Doesn’t that demand a lot of electricity and processing power?

The short answer is yes, the cost of mining is based primarily, on hardware, electricity costs, and to some degree temperature.

The problem with the Proof of work consensus is that it requires the miner to use their supercomputer to try out millions computations per second, in competition with other supercomputers around the world, to determine if the Blockchain can be updated or not.

B: Proof of Stake

The main objective of this mechanism is to allow stakeholders, the people with the most invested, or owned, in the Blockchain ecosystem to have the strongest incentives to lead in the provision of consensus solutions for a Blockchain transaction.

In simple terms:

Proof of stake consensus allows miners that have more “money”, cryptocurrency, or “skin in the game” to have a greater opportunity to mine blocks and make decisions for the network.

The process starts by the miner consuming his/her cryptocurrency — commonly referred to as the kernel — which provides privileges for updating the Blockchain which is similar to Proof of work.

However, the hashing computation in Proof of stakes is done using a limited search space where stakeholders with the greatest stakes have the ability to mine a commensurate allocation of the network, and are effectively stewards of the Blockchain system.

Think of it like: the more a miner has, the more they can get, and the more they can decide.

The one benefit of this controversial crypto-economic system is that by allowing stakeholders with incentives take charge of consensus the mechanism reduces the computing power required for consensus.

This should make the climate control kids happy, but

The main problem of this mechanism is that disadvantages other miners in the network since only the “richest” stakeholders are permitted to have control of consensus in the Blockchain.

C: Closed Consensus

In a Closed consensus mechanism certain nodes are required to put up a security deposit in order to participate in updating the Blockchain.

This consensus mechanism doesn’t require mining, and is growing in popularity in some banking and insurance segments.

The management of the consensus is done using security deposits which incentivize the validators. The “arbitrators” — conflict management nodes are the enforcers on the blockchain and the adjudicate when something is not write or if a miner is not acting fairly.

The main objective of using an arbitrator’s protocol is to enforce consensus among the autonomous nodes in the Blockchain.

If a validator authenticates a transaction which the arbitrators have considered illegitimate, then the validator losses their security deposit and they also forfeit their privileges of providing consensus in the Blockchain network in the future.

Conclusion

Now that you understand the basic essentials of Blockchain technology you should be able to distinguish very easily:

1. What is a blockchain?

2. How does bitcoin work?

3. What are the major innovations that The Ethereum blockchain brought to the technology?

4. What is a smart contract?

5. What are the different types of consensus mechanisms that power a blockchain?

Hopefully this inspires further exploration and your own personal discovery, in what everyone is talking about, and how perhaps you might be able to join in the conversation and or project/experiment.

How does the Blockchain Work? (Part 3)

What is consensus and why does it matter?

The world of Bitcoin and underlying technologies of distributed ledger, and the blockchain are experiencing rapid change and growth.

As low-trust digital-based systems gain adherents and differing use cases, developers are creating new variant blockchains to deal with the inevitable fragmentation between public, consortium and private blockchain technologies.

First, let’s note the differences between public, consortium and private blockchains.

Public — Fully decentralized and uncontrolled networks with no access permission required — anyone can participate in the consensus process to determine which transaction blocks are added. There is usually little or no pre-existing trust between participants in a Public blockchain.

Consortium — The consensus process for new transaction blocks is controlled by a fixed set of nodes, such as a group of financial institutions where pre-existing trust is high.

Private — Access permissions are tightly controlled, with rights to read or modify the blockchain restricted to certain users. Permissions to read the blockchain may be restricted or public. [1]

There is usually some degree of pre-existing trust between at least some of Private blockchain participants.

The degree of pre-existing trust that an organization requires, as well as necessary control over participant permissions, will determine what type of blockchain to use.

Different blockchain solutions have advantages and disadvantages. Take for example, the difference between how transactions are validated within each type of blockchain:

Proof of Work (PoW): About “mining” transactions utilizing a resource-intensive hashing process, which (a) confirms transactions between network participants and (b) writes the confirmed transactions into the blockchain ledger as a new block.

The accepted new block is proof that the work was done, so the miner may receive a 25 BTC (Bitcoins) payment for successfully completing the work. The problem with PoW is that it is resource-intensive and creates a centralizing tendency among miners based on computer resource capability.

Proof of Stake (PoS): About “validating” blocks created by miners and requires users to prove ownership of their “stake”[2]. Validation introduces a randomness into the process, making the establishment of a validation monopoly more difficult, thereby enhancing network security.

One problem with PoS is the “nothing at stake” issue, where miners have nothing to lose in voting for different blockchain histories, preventing a consensus from being created. There are several attempts to solve this problem underway.

Additional developments in this area hope to combine PoW with PoS to create hybrid blockchains with the highest security and lowest resource requirements.

To that end, some developers are focused on enhancing network security through ‘consensus without mining.’ [3]

Tendermint co-founder Jae Kwon has published a paper describing his firm’s concept and approach in this regard.

Existing Proof of Work and Proof of Stake protocols have various problems, such as requiring huge outlays of energy usage and increasing centralization (PoW) or participants having nothing at stake (PoS) possibly contributing to consensus disruption on mined blocks.

Kwon’s solution is twofold and does not require Proof of Work mining:

(a) A ⅔ majority of validators is required to sign off on block submission, with no more than ⅓ able to sign duplicate blocks without penalty

(b) The protocol raises the penalty of double-spend attacks to unacceptably high levels by destroying the malicious actor’s Bitcoin account values.

The algorithm is “based on a modified version of the DLS protocol and is resilient up to ⅓ of Byzantine participants.”

Kwon and his team at Tendermint hope to bring speed, simplicity and security to blockchain app development.

So, how does one decide on what type of blockchain to use and their relevancy for your company use case? [4]

Below are a few examples of different types of blockchains, depending on the organization’s greatest prioritized need:

One consideration is confidentiality. For example, in the case of a public financial blockchain, all the transactions appear on the ledgers of each participant. So while the identities of the transacting parties are not known, the transactions themselves are public.

Some companies are developing ‘supporting’ blockchains to avoid this problem, by “storing or notarizing the contracts in encrypted form, and performing some basic duplicate detection.” Each company would store the transaction data in their own database, but use the blockchain for limited memorialization purposes.

A second consideration is whether you need provenance tracking. Existing supply chains are rife with counterfeit and theft problems. A blockchain that collectively belongs to the supply chain participants can reduce or eliminate breaks in the chain as well as secure the integrity of the database tracking the supply chain.

A third example is the need for recordkeeping between organizations, such as legal or accounting communications. A blockchain that timestamps and provides proof of origin for information submitted to a case archive would provide a way for multiple organizations to jointly manage the archive while keeping it secure from individual attempts to corrupt it.

Blockchains fundamentally operate on the basis of how consensus is agreed upon for each transaction added to the ledger.

What are the benefits of each type of consensus mechanism and in which situation are they best utilized?

Proof of Work — Miners have a financial incentive to process as many transactions as quickly as possible. PoW is best utilized by high-throughput requirement systems.

Proof of Stake — Transaction Validators receive rewards in proportion to the amount of their “stake” in the network. This arguably improves network security by discouraging duplicitous attacks. PoS is best used by computing power constrained organizations.

Delegated Proof of Stake [5] — Network parameters are decided upon by elected delegates or representatives. If you value a “democratized” blockchain with reduced regulatory interference, this version is for you.

PAXOS — An academic and complicated protocol centered around multiple distributed machines reaching agreement on a single value. This protocol has been difficult to implement in real-world conditions.

RAFT — Similar to PAXOS in performance and fault tolerance except that it is “decomposed into relatively independent subproblems”, making it easier to understand and utilize.

Round Robin — Utilizing a randomized approach, the round robin protocol requires each block to be digitally signed by the block-adder, which may be a defined set of participants. This is more suited to a private blockchain network where participants are known to each other.

Federated Consensus — Federated consensus is where each participant knows all of the other participants, and where small sets of parties who trust each other agree on each transaction and over time the transaction is deemed valid. Suitable for systems where decentralized control is not an imperative.

Proprietary Distributed Ledger — A PDL is one where the ledger is controlled, or proprietary, to one central entity or consortium. The benefits of this protocol is that there is already a high degree of pre-existing trust between the network participants and agreed-upon security measures. Suitable for a consortium or group of trading partners, such as supply chains.

PBFT — In a PBFT system, each node publishes a public key and messages are signed by each node, and after enough identical responses the transaction is deemed valid. PBFT is better suited for digital assets which require low latency due to high transaction volume but do not need large throughput.

N2N — Node to node (N2N) systems are characterized by encrypted transactions where only the parties involved in a transaction have access to the data. Third parties such as regulators may have opt-in privileges. Suitable for use cases where a high degree of transaction confidentiality is required.

The above list represents the current major consensus mechanisms in operation or from research.

Due to the initial visibility of Bitcoin, the financial services industry has been early in researching the possible uses of consensus mechanisms to streamline operations, reduce costs and eliminate fraudulent activity.

The multi-trillion dollar global financial services industry is really composed of many different sectors, from lending to smart contracts, trading execution, letters of credit, insurance, payments, asset registration, regulatory reporting and more.

For example, the process of securing a letter of credit, which is an important import/export trading service, would likely utilize a ‘consortium’ approach to achieving transaction consensus.

In August, 2016 a banking consortium, R3CEV, successfully designed and executed trading smart contracts. These types of contracts could then be applicable to accounts receivable invoice factoring and letter of credit transactions.

For the use case example of cross border remittances, which would involve many individuals on both sides of the transaction, a ‘public’ consensus mechanism would likely be a relevant choice.

Since remittances would need to have a relatively short time latency for transaction completion, a solution involving a Proof of Stake approach with its low resource requirement to validate transactions along with potentially higher security, would be compelling.

In sum, the state of blockchain development is rapidly gaining speed worldwide, yet there is much work to be done.

Numerous Global 2000 companies led by their technology executives and consultants are beginning to participate in development and testing of this revolutionary technology sector.

Organizations that begin first-hand learning about the power of blockchain technologies will have increased opportunity to lead their industry.

How does the Blockchain Work? (Part 4)

What’s the difference between a private, public, and consortium blockchain?

Public, Private, Consortium Blockchain

Understanding the differences between Private, Public and Consortium Blockchains is important.

As financial institutions begin to explore the possibilities of blockchain technology, they are coming up with systems that complement their existing business models. A private or a consortium blockchain platform, as opposed to the public platform that Bitcoin uses, will allow them to retain control and privacy while still cutting down their costs and transaction speeds.

In fact, this private system will have lower costs and faster speeds than a public blockchain platform can offer.

Blockchain purists aren’t impressed. A private platform effectively kills their favorite part of this nascent technology: decentralization.

They see the advent of private blockchain systems as little more than a sneaky attempt by big banks to retain their control of financial markets.

In a way, they’re correct.

Though the evil plot narrative is a bit much. If big banks can utilize a form of blockchain technology that revolutionizes finance, and if they are willing and able to pass these benefits onto their customers, then it is hardly an evil plot.

It is just good business.

Vitalik Buterin said it best:

“the idea that there is ‘one true way’ to be blockchaining is completely wrong headed, and both categories have their own advantages and disadvantages.”[1]

Let’s take a deeper look at what these might be.

Public Blockchain

A Blockchain was designed to securely cut out the middleman in any exchange of asset scenario. It does this by setting up a block of peer-to-peer transactions. Each transaction is verified and synced with every node affiliated with the blockchain before it is written to the system.

Until this has occurred, the next transaction cannot move forward. Anyone with a computer and internet connection can set up as a node that is then synced with the entire blockchain history.

While this redundancy makes public blockchain extremely secure, it also makes it slow and wasteful.

The electricity needed to run each transaction is astronomical and increases with every additional node. The benefit is every transaction is public and users can maintain anonymity.

A public blockchain is most appropriate when a network needs to be decentralized.

It is also great if full transparency of the ledger or individual anonymity are desired benefits. Costs are higher and speeds are slower than on a private chain, but still faster and less expensive than the accounting systems and methods used today.

This is a good trade-off for a cryptocurrency like Bitcoin.

Security is key to their users, a decentralized network is at the heart of the project and their competitors in the finance industry are still significantly more expensive and slower than a public blockchain network despite its slowness when compared to a private blockchain.

Private Blockchain

Private blockchain lets the middleman back in, to a certain extent.

I believe the saying goes: better the devil you know, than the devil you dont’tknow.

The company writes and verifies each transaction. This allows for much greater efficiency and transactions on a private blockchain will be completed significantly faster. Though it does not offer the same decentralized security as its public counterpart, trusting a business to run a blockchain is no more dangerous than trusting it to run a company without blockchain.

The company can also choose who has read access to their blockchain’s transactions, allowing for greater privacy than a public blockchain.

A private blockchain is appropriate to more traditional business and governance models, but that isn’t a bad thing. Just because it is unlikely to revolutionize our world, doesn’t mean it can’t play a role in making the world better.

Competition is key to developing the most useful products. Traditional financial institutions have long held a monopoly — technically, an oligopoly — over the industry.

Their outdated products and services are a direct result of this power.

Using a privately run version of blockchain technology can bring these organization into the 21st century.

A number of our governance institutions are old and outdated as well.

Like finance, our government is not subject to competition. Adoption and integration will likely be slower in this sector, but if and when blockchain technologies are adopted they will cut billions of dollars of behind the scenes spending.

Imagine a truly secure online voting system. No more poll workers, voting booths, paper ballots, paid counters or organizers with cushy salaries. What’s more, the barriers to voting will be greatly reduced and we will likely see an increase in turnout.

This could be accomplished with a public design, but most governments are unlikely to decentralize control and security, so a vetted private system greatly increases the chance of adoption.

Consortium Blockchain

Consortium blockchain is partly private. THere has been some confusion about how this differs from a fully private system. Vitalik Buterin provides a pretty straightforward definition:

“So far there has been little emphasis on the distinction between consortium blockchains and fully private blockchains, although it is important: the former provides a hybrid between the ‘low-trust’ provided by public blockchains and the ‘single highly-trusted entity’ model of private blockchains, whereas the latter can be more accurately described as a traditional centralized system with a degree of cryptographic auditability attached.”[2]

Instead of allowing any person with an internet connection to participate in the verification of transactions process or allowing only one company to have full control, a few selected nodes are predetermined.

A consortium platform provides many of the same benefits affiliated with private blockchain — efficiency and transaction privacy, for example — without consolidating power with only one company.

You can think of it as trusting a council of elders. The council members are generally known entities and they can decide who has read access to the blockchain ledger.

Consortium blockchain platforms have many of the same advantages of a private blockchain, but operate under the leadership of a group instead of a single entity.

This platform would be great for organizational collaboration.

Imagine central banks coordinating their activities based on international rules of finance. Or the United Nations outsourcing their transactional ledger and voting system to blockchain, allowing each country to represent a verifying node.

The possibilities are endless.

Blockchain Use Cases Moving Forward

Outside of finance and governance, we are only just beginning to explore the uses of private and consortium blockchain technologies. If blockchain’s main use is to cut out the middle man and allow for peer-to-peer transactions, it is invariably going to be disliked by those middlemen.

This is where we are now.

Companies scrambling to make their business models safe from this disruptive technology.

But this isn’t the most exciting use of blockchains.

Ideas that do unprecedented things with this technology, instead of just applying it to existing models, will be where this technology shines.

If I could list exactly how and where these breakthroughs will take place, I would be out making them a reality myself. Rest assured, they do exist.

Let’s not forget that the pieces of technology that built Bitcoin were around long before the cryptocurrency was designed.

Advertisements

The CLOUD Act and Google: How it affects your data

What is the CLOUD Act and what does it mean when it comes to my privacy as a Google customer?

The CLOUD Act — Clarifying Lawful Overseas Use of Data — is a set of regulations handling how data stored in one country can be accessed by an entity in a different country. It was signed into law on March 23, 2018 as part of the Omnibus Spending Bill.

It’s been praised by technology companies and a joint letter from Apple, Facebook, Google, Microsoft, and Oath (Yahoo!) lending support to the bill was published on February 6, 2018. it states, in part:

The new Clarifying Lawful Overseas Use of Data (CLOUD) Act reflects a growing consensus in favor of protecting Internet users around the world and provides a logical solution for governing cross-border access to data. Introduction of this bipartisan legislation is an important step toward enhancing and protecting individual privacy rights, reducing international conflicts of law and keeping us all safer.

But privacy and civil rights organizations have a different opinion of the legislation. The ACLU had this to say:

The CLOUD Act represents a major change in the law — and a major threat to our freedoms. Congress should not try to sneak it by the American people by hiding it inside of a giant spending bill. There has not been even one minute devoted to considering amendments to this proposal. Congress should robustly debate this bill and take steps to fix its many flaws, instead of trying to pull a fast one on the American people.

The Electronic Frontier Foundation has a list of objections as well:

  • Includes a weak standard for review that does not rise to the protections of the warrant requirement under the 4th Amendment.
  • Fails to require foreign law enforcement to seek individualized and prior judicial review.
  • Grants real-time access and interception to foreign law enforcement without requiring the heightened warrant standards that U.S. police have to adhere to under the Wiretap Act.
  • Fails to place adequate limits on the category and severity of crimes for this type of agreement.
  • Fails to require notice on any level – to the person targeted, to the country where the person resides, and to the country where the data is stored. (Under a separate provision regarding U.S. law enforcement extraterritorial orders, the bill allows companies to give notice to the foreign countries where data is stored, but there is no parallel provision for company-to-country notice when foreign police seek data stored in the United States.)
  • The CLOUD Act also creates an unfair two-tier system. Foreign nations operating under executive agreements are subject to minimization and sharing rules when handling data belonging to U.S. citizens, lawful permanent residents, and corporations. But these privacy rules do not extend to someone born in another country and living in the United States on a temporary visa or without documentation.

The two sides seem to take the language in the CLOUD Act very differently. That’s to be expected with almost any legal document, and most bills introduced to Congress are written in the same type of language. It purposefully leaves things open to the interpretation of the reader, and in the case of laws, the enforcing body. We all will have our own opinion on the bill, and that’s a healthy discussion to have. But it’s important to know what this means for your data stored on Google’s servers.

Why would Google support this?

It’s important to remember that organizations like the ACLU and EFF exist to examine the worst-case scenario surrounding any rules or laws that govern our personal data. They help create a balance so that courts and legislators can make informed rulings and seeing their objection to the CLOUD Act isn’t a surprise because it makes some major changes to the existing laws. It’s very difficult for a foreign government to gain access to data saved on a U.S. server and for the U.S. government to obtain data stored on a foreign server because the laws vary from country to country.

An example of this in action is currently happening, as the U.S. Supreme Court is deciding if Microsoft needs to turn over data stored on an Irish server that the Department of Justice wants as evidence in a case that dates back to 2013.

Companies like Google would rather see a single set of rules adopted by the U.S. and many other countries that they do business in that might prevent this sort of costly hearings and procedures. They feel the language in the CLOUD Act serves to provide access to our data when a genuine need arises but also protects our privacy against requests that don’t show a legitimate need.

A set of universal laws that protect our privacy is a great idea as long as the laws are sound and enforced.

Civil rights organizations would also like to see a single set of rules adopted around the world, but do not think the CLOUD Act sufficiently protects our information from foreign governments. They take issue with how it changes the judicial review process and the ways it may circumvent the 4th Amendment to the U.S. Constitution, as well as how the bill was introduced and packaged into a larger spending bill which won’t have the scrutiny and publicity a change like this deserves before it’s written as law.

Taken at face value, both sides here seem to be correct. That’s because both sides are fulfilling their intended purposes. Google’s legal team and privacy experts want a simple set of rules that apply in every country it operates in and thinks that circumventing a court hearing or obtaining multiple individual warrants can be done in a way that still protects it’s users personal data under the CLOUD act. The ACLU and EFF are against anything that circumvents a judicial process for each individual request and they feel that the current system provides better privacy standards. It’s important for lawmakers to hear both arguments.

What does this mean for me and my data?

There is no language in the CLOUD act that changes the way Google stores your data or the data it can collect. Nothing there strips away the protections of encryption nor does it prevent you from deleting your data from Google’s servers at any time. The only thing the CLOUD act affects is how your data stored on a server in your country, can be shared with another nation’s government. But that is something we all should be concerned about, too, so let’s look at some specifics.

Are my civil liberties being protected?

The CLOUD act requires the Secretary of State and the Attorney General of the United States to certify that any country entering into the CLOUD ACT “affords robust substantive and procedural protections for privacy and civil liberties.” Some specifics are mentioned in the bill to protect our rights as Americans. They include:

  • Protection from arbitrary and unlawful interference with privacy
  • Fair trial rights
  • Freedom of expression, association, and peaceful assembly
  • Prohibitions on arbitrary arrest and detention
  • Prohibitions against torture and cruel, inhuman, or degrading treatment or punishment.

This means any country that participates in the CLOUD act can’t trample the basic civil rights afforded to us as citizens of the U.S. — and that rights of citizens in other countries can’t be trampled by the U.S. government. Protections against a foreign government requiring Google to place a backdoor into Android or Chrome are also in place under the CLOUD act and that Google can’t be asked by any government to perform surveillance on us while we use their products.

Does the CLOUD act give the executive branch complete control over our data rights?

No. While it does allow the State Department and Attorney General’s office to make agreements with foreign nations there is some Congressional oversight built in. Congress will have the power to:

  • Review new bilateral agreements for up to 180 days.
  • Review changes to existing agreements for up to 90 days.
  • Require written certification and explanation for how countries pass certification.
  • Fast-track disapproval of bilateral agreements.

It also states that a surveillance order issued by any member country be individually based and “subject to review or oversight by a court, judge, magistrate, or other independent authority,” and that this review must be “prior to, or in proceedings regarding, enforcement of the order.”

It would be better to have these protections in place as part of the way agreements between participating countries are made, but they are there, and in language that’s surely enforceable should a country be found to be overstepping its bounds.

Does the CLOUD act make it easier for foreign nations to access my U.S.-based data?

Yes. The CLOUD act removes many of the obstacles currently in place when another country wants your data stored on a Google server inside the United States. This is where civil rights organizations and Google disagree on the merits of the law.

Because of how any data requests must go through the court system, then be subject to appeal or approval from a higher court, countries are forming their own laws that try and force companies like Google to hand over data without any court involvement if the company wants to do business there out of frustration with the process. The U.S. also tries to claim that U.S. law requires a U.S. company to hand over data even when it’s hosted outside the country like we’re seeing in the Microsoft case presented to the Supreme Court.

Some countries provide civil liberties that are equal or better than what the Constitution offers, but others do not.

The CLOUD act is designed to stop these laws from being enacted and enforced by building a process all countries can agree on and adhere to when it comes to requests for our private data. This is where Apple, Google, Microsoft and other tech companies see the benefit of it. They will know what the laws are and how to follow them in all the countries that participate instead of being subject to individual laws or fighting them in courts.

Civil rights organizations take issue that the CLOUD act can force data hosted inside the U.S. to be handed to another nation without being subject to our existing privacy laws. Some countries provide civil liberties that are equal or better than what the Constitution offers, but others do not. They feel that your data hosted in the U.S. should be protected by your rights as a U.S. citizen and not subject to laws and rights another country observes no matter what the review or admittance process entails.

Does the CLOUD act give foreign countries more power to surveil U.S. citizens and target their data for collection?

No, and yes. Broader power is granted for intelligence gathering but there are restrictions and rules in place that cover any wiretapping or surveillance.

  • Foreign governments are “explicitly forbidden from surveilling a U.S. person directly or indirectly”.
  • Surveillance orders must be of a fixed and of limited duration.
  • Surveillance can only happen when it has been shown to be “reasonably necessary” and there is no other way to get the information.

When collecting data for approved cases, there are rules in place that aim to protect our individual rights:

  • Direct targeting of a U.S. citizen’s data by non-U.S. governments is prohibited.
  • Asking a CLOUD Act certified country to target a U.S. persons’ data is prohibited.
  • The targeting a non-U.S. persons’ data for the purpose of collecting a U.S. persons’ data is prohibited. (A country can’t target me to see the conversations you and I have in Facebook Messenger, for example.)
  • The “dissemination of a U.S. persons’ data” is prohibited unless there is evidence of a serious crime presented.

There is a lot of room for legal maneuvering in these regulations, which leads us to the biggest question — how will this be enforced? Who will be there to make sure France (for example) follows the laws and regulations about collecting my data inside the U.S.? That’s worrisome. Even more so when you replace France with Afghanistan, or if you live in Europe and replace France with the United States. Current laws are in place to protect our data and we’ve grown accustomed to having them. the CLOUD act would replace many of those protections.

Do I need to worry, and should I delete all of my data and go dark?

I’m not a legal expert so I can’t form an opinion on the legality of the CLOUD act. That’s what we elect officials to do. But I can express a few thoughts on it all. I’m of the opinion that my data stored in the U.S. is protected under the laws of the U.S. and secured with my rights as a U.S. citizen regardless of what France (or Afghanistan) thinks of those protections.

Guaranteed liberties like the 4th amendment (the protection against unreasonable search and seizure defined as an individual right of every U.S. citizen) or its equivalent in other countries should always apply and supersede any type of unilateral act between governments. Every instance where my privacy is to be breached is deserving of its own review in the U.S. courts, especially if I’m not proven guilty of any serious crimes.

My data is deserving of a review process every time a person or nation requests access. So is yours.

But I also see the value that Google sees in the CLOUD act. A legitimate set of rules that apply across the board for all member nations could be a great thing; not only to save money and time in courts but so that I know in advance how my data is protected both inside and outside of the U.S.

We should be able to trust our elected officials to make the right choices, and if you do then there isn’t much to be concerned about here. It seems that Google trusts the “right” way to guarantee our privacy will be put in place, as does Apple and Microsoft. These three companies may have a very different set of offerings to present to us, but one thing they all have in common is the willingness to fight to protect our data. That’s a good reason to assume the sky isn’t falling.

The ACLU and EFF, as well as other privacy and civil rights groups, have also done a great job of making sure we know when our rights may be subject to abuse. We should pay attention to their warnings even if we think they are reaching the worst conclusion. This is a good reason to be against the CLOUD act in any form.

Right now, all we can do is watch the process in action and hope everyone involved is thinking about our individual rights when they make their decision. Once that decision is reached, we can decide how to react. What’s most important is that we know and understand when the laws surrounding our personal data are going to be changed, and what the consequences may be.

Are neural networks the future of AI? How important are they?

Scott E. Fahlman, Professor Emeritus, Carnegie Mellon, LTI and CSD

Image result for neural network

Complicated question, and one for which there is no real consensus among the experts.  I’ll very briefly state my own opinion, but that is not necessarily a majority view.

For what it’s worth, I’m one of the relatively few researchers who have done serious work on both neural-net learning algorithms (including some that did a kind of deep learning 25 years ago) and on symbolic methods for human-like, common-sense knowledge representation, reasoning, planning, and language understanding.

I believe that deep-learning neural nets (but not necessarily the algorithms being used now) will play a very important role in the future of AI. If we want to emulate human capabilities, I think that the neural nets will pretty much take over the lower-level parts of sensory-motor processing, and speech/language understanding, probably up to and including the learning of sequential word patterns and syntax.

Roughly, this is the stuff that we humans do without being aware of what is going on or how we learned it: standing, walking, reaching and grasping, throwing; picking out the words in a noisy stream of speech; recognizing objects, their parts, and spatial relations in a scene.

I don’t believe that neural networks, as currently understood, will take over higher-level conscious thought and planning (including creative planning and design); the symbolic parts of knowledge representation and inference; and language understanding/generation tasks that involve meaning. We will need symbolic representations for these things. I will be surprised if distributed “thought vectors” are adequate representations for these tasks.

In the human, it is pretty clear that this higher-level, more symbolic stuff must also be implemented in some sort of neural network — that’s all there is in the brain — but it these neurons are not operating like current feed-forward or generative neural-net models. Instead, these networks are functioning more like conventional computers that manipulate symbols, but with some massively parallel symbolic search and inference capabilities built in.

The neural-net and symbolic levels have to work together, and what happens at the interface is a very interesting area for investigation.  It’s pretty clear that the lower-level pattern-recognition parts are influenced by our expectations, some of which come from higher-level reasoning; it’s also pretty clear that the pattern-recognition and pattern-learning parts must be able to cause the creation of new symbols and relations that are accessible to the higher-level symbolic machinery.

By the way, my use of the terms “higher-level” and “lower-level” is not a value judgement, just a shorthand for the way most people classify certain mental functions.  Some of the “highest level” cognitive tasks, such as chess and calculus, were among the first things that AI researchers solved, while “lower-level” tasks such as manual dexterity and recognizing objects from images are only now starting to make real progress towards human-like performance.

Again, that is just one researcher’s best guess about where things are headed in AI.  Read what other researchers are saying and you will get a variety of other viewpoints and guesses.

Amazon ready to Disrupt the Market

Opinions expressed by Forbes Contributors are their own.

The Platform as a Service (PaaS) market is going through metamorphosis. A key driver of this change is the container revolution, led by Docker. Every PaaS vendor in the market has refactored its platform for containers. On the other hand, the combination of orchestration tools such as Kubernetes, Mesos, and Docker, is becoming an alternative to traditional PaaS. The line between container orchestration and PaaS is getting blurred. For enterprises and decision makers considering PaaS, the current market landscape looks complex and confusing. Amidst all this chaos, one vendor who is quietly redefining PaaS is Amazon Web Services.

Amazon EMR

 

Having invested heavily in the core building blocks of infrastructure – compute, storage, and networking; Amazon has been steadily moving up the stack to focus on platform services. From its vantage point, AWS has visibility into top customer use cases and deployment scenarios. By carefully analyzing what customers run in its infrastructure, AWS is building new managed services that are quickly becoming an alternative to self-hosted workloads. Amazon RDS, AWS Directory Services, Amazon Elastic File System, Amazon WorkMail, Amazon WorkDocs, and Amazon EC2 Container Service are a few examples of these services. AWS wants customers to sign-up for its managed services instead of following the DIY approach. In its current form, AWS can support everything a small and medium business needs. From hosted desktops to file sharing to collaboration to backup and archival, Amazon has it all. Beyond enterprise and business applications, it is now eyeing developers by offering a parallel universe of application lifecycle management in the cloud. The new family of code management services such as AWS CodeDeploy, AWS CodeCommit, AWS CodePipeline, handle the entire lifecycle of a cloud-native application. Amazon is in the process of building a brand new PaaS that is very different from the rest.

Amazon API Gateway – an application programming interface management layer – is the latest addition to the AWS application services portfolio. Though it might just look like another service from AWS, this has the potential to become the cornerstone of AWS’ PaaS strategy. Amazon is calling this service the “front door” for applications to access data, business logic, and functionality from back-end services. API Gateway is another classic customer workload that became a managed service on the AWS cloud. So, how does this service enable Amazon to disrupt the PaaS market?

Last year at the AWS re:Invent Conference, Werner Vogels unveiled a killer microservices platform called AWS Lambda. In a Gigaom Research report entitled Why AWS Lambda is a Masterstroke from Amazon, I analyzed the importance of this service. What’s special about Lambda is that it is a true NoOps platform. Developers bring their autonomous code snippets that get invoked by an external event. Since its inception, AWS has been regularly adding Lambda hooks for popular services like S3, DynamoDB, Kinesis, and SNS. It recently added Java language and JDK to this microservices platform. Though it was tempting to port the bulk of the business logic and workflow from monolithic apps to AWS Lambda, the service didn’t support exposing the code snippets as REST endpoints. Developers had to rely on service hooks to indirectly trigger Lambda functions.

One of the most powerful aspects of the new Amazon API Gateway, is its integration with AWS Lambda. Developers can upload code snippets to Lambda and expose it as a standard REST endpoint hosted by the API Gateway, which essentially becomes the facade to the microservices platform. This service eliminates the need to spin up an EC2 instance that runs business logic exposed as an API. What’s more? Developers can point and click to configure an API key, throttling, bursting, caching and even adding a custom domain. Finally, they can also generate native SDKs of their APIs for Android, iOS, and JavaScript. This combination of AWS Lambda and API Gateway becomes a powerful microservices platform without the tax of scheduling, orchestration, monitoring, logging and security. Both API Gateway and AWS Lambda are elastic, enabling the developer to focus on the logic and code. Through the integration of CloudTrail and CloudWatch, performance metrics and logs are instantly available. Microservices hosted in AWS Lambda can consume AWS SDK to communicate with other services such as Amazon RDS and Amazon DynamoDB. This deployment topology makes applications highly available, scalable and secure with no operations required. Deploying the same applications on a traditional PaaS involves quite a bit of configuration and management.

 

But API endpoints and code do not make an application complete. It needs an interface to become web applications and mobile apps. Since the heavy-lifting is offloaded to AWS Lambda, all that the developer needs to do is to host the web application that consumes the API exposed by the API Gateway. This is where Amazon S3’s web hosting feature comes in handy. Designers and developers can build beautiful web interfaces based on Bootstrap, AngularJS, or other JavaScript frameworks. Since API Gateway supports the generation of JavaScript SDK, it can be consumed in static web applications hosted in Amazon S3. The same API can be targeted by native Android and iOS applications. For authentication and security, the application can be integrated with Identity and Access Management (IAM). This configuration completely avoids the need to spin EC2 instances dedicated to hosting applications. The combination of S3, API Gateway, and AWS Lambda delivers scale without the need for administration.

 

CloudHealth Technologies!

We’re in the midst of one of the most profound transitions in IT history: the movement to the cloud. While the benefits – such as lower upfront costs, reduced management requirements, on-demand scaling – are widely understood, managing, optimizing and securing cloud infrastructure is a different story. It can actually be substantially more challenging than traditional data center infrastructure because of its dynamic nature.

As a business begins to utilize multiple cloud providers, as well as their own data centers, these problems compound. Pressure mounts on engineering teams to automate processes and lower costs. Leaders shift valuable engineering resources away from core product development to cloud maintenance. Ops teams write programs to help automate instance purchasing and management, but cloud providers constantly change their pricing structure and technology infrastructure, rendering internally built technologies obsolete.

Welcoming CloudHealth Technologies!

This is why we invested in CloudHealth.

CloudHealth is a Cloud Service Management platform that takes the pain out of managing cloud deployments and puts the power back in the hands of business users (in particular, the CFO & CIO). The platform provides customers with a centralized console for users to manage their hybrid and multi-cloud infrastructure. It integrates directly with cloud providers’ infrastructure, enabling customers to optimize and automate instance purchasing to take advantage of pricing changes, allows a business user to automate all rules, policies and governance, and gives security professionals visibility into real-time risks.

In reducing complexity, CloudHealth adds immediate ROI for all stakeholders. Engineers can return their focus to the core product, finance teams are able to cut costs and gain visibility into usage by team and expense bucket, and CIOs can improve and optimize governance.

CloudHealth Technologies’ exceptionally strong growth and unit economics demonstrate the unique value customers achieve with the product. It is emerging as the clear leader in the category, has some of the strongest retention cohorts we’ve seen and is extremely well positioned to capitalize on the continued growth and complexity of hybrid and multi-cloud environments. Importantly, while the business has scaled rapidly, culture has scaled thoughtfully (just check out the Glassdoor reviews).

The CloudHealth Technologies team should be incredibly proud of what they’ve accomplished to date. In particular, Dan Phillips (C-Founder & CEO), Joe Kinsella (Co-Founder & CTO) and Larry Begley (CFO & one of the first institutional investors while a GP at .406 Ventures) deserve tremendous credit for the team they’ve assembled and the business they’ve built.

We are thrilled to welcome the entire CloudHealth Technologies team to the Kleiner Perkins family!