Design of Reconfigurable Multi-Operand Adder for Massively Parallel Processing

Abstract

The paper presents a systematic study and implementation of a reconfigurable combinatorial multi-operand adder for use in Deep Learning systems. The size of carry changes with the number of operands and hence a reliable algorithm to estimate exact number of carry bits is needed for optimal implementation of a reconfigurable multi-operand adder. A combinatorial multi-operand adder can be faster compared to a sequential implementation using a two operand adder. Use cases for such adders occur in modern processors for deep neural networks. Such processors require massively parallel computing resources on chip. This paper presents a method to estimate the upper bound on the size of carry. A method to compute the exact number of carry bits required for a multi-operand addition operation. A fast combinatorial parallel 4-operand adder module is presented. An algorithm to reconfigure these adder modules to implement larger adders is also described. Further, the paper presents two compact but slower iterative structures that implement multi-operand addition, iterating with one column at a time till the entire word is covered. Such serial/iterative operations are slow but occupy small space while parallel operations are fast but use large silicon area on chip. Interestingly, the area-to-throughput ratio of two architectures can tilt in favor of slower, smaller and large number units instead of the fewer numbers of fast and large compute units. A lemma presented in the paper may be used to identify the condition when such tilt occurs. Potentially, this can save silicon space and increase the throughput of chips for high performance computing. Simulation results of a 16 operand adder and using an set of 4-operand adders for use in neural networks have been presented. Simulation results show that performance gain improves as the number of operations or operands increases.