Tech Briefing: Artificial neural networks

Despite each neuron having little computing power, networks composed of many units can solve complex problems in efficient ways. 

The similarity between these artificial networks, and how neural networks actually work, is striking enough that neuroscientists and computer scientists intersect in their research areas.

Biological and coded brains share the capability to evolve and improve as they solve problems. Like a child practicing math homework, an ANN learns from examples. The more examples the machine faces, the more effective it becomes at solving problems efficiently. In the simplest of terms, this is the outcome of ANNs.

The analogy roughly stops here. ANNs replicate neurons as nodes of numbers, whereby biological neurons use chemical and electrical signals to transmit information. Thus the use of ANNs does not automatically entail machine sentience or “intelligence.”

The simplest building block of ANNs is the artificial neuron, or node, theorised by Frank Rosenblatt in 1958. By the 1970s, ANNs were long dismissed as impractical or impossible. Most computer scientists at the time believed that machines need rules and strict orders to function. Geoff Hinton disagreed. Hinton believed that if the brain can learn from scratch, so can machines. 

ANNs are, at their logical core, a decision tree. Put simply, each node has an input, a hidden layer, and an output. If the output of the calculation is higher than a given threshold, the node passes the information to the next layer. Once all outputs are calculated, they are combined into a formula that reflects the learnings from that training phase. This mathematical formula enables machines to make decisions based on other inputs. An ANN with more than three layers can be referred to as a deep learning system.

Multiple types of ANNs enable different types of learning: 

• “Supervised learning” uses labeled examples to teach the machine. Imagine the machine is given many photos of apples labeled “Apple.” The machine could then learn to recognise unlabelled photos of apples in a dataset and label them with a high level of accuracy. 
• “Unsupervised learning” allows a machine to discover patterns in a dataset without direction. The learning dataset may have uncategorised photos of apples, oranges and pears; the machine could notice that there are three categories of items, and may be able to sort new data in those three categories. 
• The third type of learning is “reinforcement learning.” In this case, the machine is given input data with no solution, and the output of the network is given a score. Over many iterations, the machine will seek to maximise this score, and learn how to do so in the process. This powerful method enabled machines to teach themselves how to drive or how to play games, for example. 

Overall trends

The versatility and ubiquity of ANNs in artificial intelligence applications has driven spectacular growth in technology use in virtually every facet of modern life. 

The ability of machines to learn has newsworthy applications in nearly every industry—I cannot think of one where it would not. ANNs have also fuelled a wave of hyper-automation: They have been hailed as the next doctors, with many breakthroughs in diagnostics and medical imaging, including classifying tumors and the discovery of COVID-19 drugs and vaccines. They have also found applications in banking, finance and investment, real estate valuation, food science, crime prevention, and as an instrument in global policymaking, among others. We certainly notice a rush to patent marketable solutions, and our engine detected significant investment interest peaking in 2018, together signalling that this trend is here to stay. 

There are, however, significant drawbacks to over-reliance on automated decision making, and cracks are forming as machines are tested in the real world. In many cases, human medical specialists still outperform machines, and errors in automated surveillance can cause deep hurt and reinforce social injustice.

Top research institutions

Looking broadly at the past five years of academic publications that mention ANNs, the top five publishing research institutions are located in China, with Tsinghua University taking the lead. Outside China, Harvard University in the US is number one, followed closely by Anna University in India. Research institutions in China and India take the lead in the number of publications directly related to ANNs. The US places third.

Top research funders

Academic research is largely funded by public institutions, such as the National Natural Science Foundation of China, the European Commission, and the US-based National Science Foundation. US government institutions are among the biggest investors in long-term research. The European Commission and the European Research Council follow. 

Top patent holders

Patents related to ANNs grew significantly in the past five years. South Korea, China and the US are the top three countries that hold the most patents, trailed by Germany and India. The largest patent holder is Samsung, followed by Huawei and IBM.