How to efficiently add BigDecimals

Anyone who deals with monetary values knows that double/float won't cut the mustard and if you deal with prices and FX rates, then BigDecimal is the only real option.

This comes with a lot of potential issues, BigDecimal methods do not handle null very well (i.e. not at all) and sometimes a bug crops up because BigDecimal returns new instances.

	BigDecimal bd1 = new BigDecimal(10);
	bd1.add(new BigDecimal(20)); // oops
	// bd1 is still 10
	// and the result of the addition is... lost
	// USING Total From ObjectLabKit
	Total total = new Total(new BigDecimal(10));
	total.add(new BigDecimal(20));
	// total is now 30
	System.out.println(total.getTotal());

view raw bug.java hosted with ❤ by GitHub

So the ObjectLabKit Util package will help, but here is a question for you... what is an efficient way to sum a list of BigDecimal coming from a Class.

	private static class Test {
	private final BigDecimal value;
	public Test() {
	value = null;
	}
	public Test(final int value) {
	super();
	this.value = BigDecimal.valueOf(value);
	}
	public BigDecimal getValue() {
	return value;
	}
	}

view raw Test.java hosted with ❤ by GitHub

Assume that we have a list of 500 Test instances and that we need to sum the Test.value and that value could be null.

We shall run the test 1,000 times.

Option 1: Use Total in a for loop

	private static void useTotal(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+Loop");
	final Total total = new Total();
	for (final Test t : list) {
	total.add(t.getValue());
	}
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", total.getTotal(), mon.getLabel(), mon.getLastValue(),
	mon.getAvg(), mon.getMin(), mon.getMax());
	}

view raw calc-usingTotal.java hosted with ❤ by GitHub

Option 2: Use Total with java8 forEach

	private static void useTotalAndForEach(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+forEach");
	final Total total = new Total();
	list.stream().forEach(t -> total.add(t.getValue()));
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", total.getTotal(), mon.getLabel(), mon.getLastValue(),
	mon.getAvg(), mon.getMin(), mon.getMax());
	}

view raw calc-usingTotalAndForEach.java hosted with ❤ by GitHub

Option 3: Use Total and java8 map()

	private static void useTotalAndMap(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+Map");
	final Total total = new Total();
	final Iterator<BigDecimal> it = list.stream().map(Test::getValue).iterator();
	while (it.hasNext()) {
	total.add(it.next());
	}
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", total.getTotal(), mon.getLabel(), mon.getLastValue(),
	mon.getAvg(), mon.getMin(), mon.getMax());
	}

view raw calc-usingTotalAndMap.java hosted with ❤ by GitHub

Option 4: Use Java8 map and reduce

	private static void useMapAndReduce(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+reduce1");
	final BigDecimal reduce = list.stream().map(Test::getValue).reduce(BigDecimal.ZERO, (a, b) -> b != null ? a.add(b) : a);
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
	mon.getMin(), mon.getMax());
	}

view raw calc-usingMapAndReduce.java hosted with ❤ by GitHub

Option 5: Use Java8 map, reduce and accumulator

	private static void useReduceAndAccumulator(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+reduce2");
	final BigDecimal reduce = list.stream().map(Test::getValue).reduce(BigDecimal.ZERO, BigDecimalAccumulator.INSTANCE);
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f count %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
	mon.getMin(), mon.getMax());
	}

view raw calc-useMapReduceAccumulator.java hosted with ❤ by GitHub

Option 6: Use Java8 and home-made Collector

	private static void useCollector(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+collect");
	final BigDecimal reduce = list.stream().map(Test::getValue).collect(new ToTotalCollector()).getTotal();
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
	mon.getMin(), mon.getMax());
	}
	public static class ToTotalCollector implements Collector<BigDecimal, Total, Total> {
	@Override
	public Supplier<Total> supplier() {
	return Total::new;
	}
	@Override
	public BiConsumer<Total, BigDecimal> accumulator() {
	return Total::add;
	}
	@Override
	public Function<Total, Total> finisher() {
	return Function.identity();
	}
	@Override
	public BinaryOperator<Total> combiner() {
	return (t1, t2) -> {
	t1.add(t2);
	return t1;
	};
	}
	@Override
	public Set<Characteristics> characteristics() {
	return Collections.unmodifiableSet(EnumSet.of(IDENTITY_FINISH, CONCURRENT));
	}
	}

view raw calc-usingCollector.java hosted with ❤ by GitHub

Option 7: Use Java8 and ObjectLabKit Calculator

	private static void useCalculator(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+calc");
	final BigDecimal reduce = Calculator.sum(list, Test::getValue);
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
	mon.getMin(), mon.getMax());
	}
	private static class Calculator {
	public static <T> BigDecimal sum(final Collection<T> collection, final Function<T, BigDecimal> mapper) {
	return collection.stream().map(mapper).reduce(BigDecimal.ZERO, (l, r) -> r != null ? l.add(r) : l);
	}
	}

view raw calc-usingCalculator.java hosted with ❤ by GitHub

Option 8: Use Java8 and Parallel Stream

	private static void useParallelCollector(final List<Test> list) {
	final Monitor mon = MonitorFactory.start("Total+collecP");
	final BigDecimal reduce = list.parallelStream().map(Test::getValue).collect(new ToTotalCollector()).getTotal();
	mon.stop();
	System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
	mon.getMin(), mon.getMax());
	}

view raw calc-usingParallelStream.java hosted with ❤ by GitHub

So what are the results?

On my 2012 MacBook Pro for a list of 500 Test instances.

Algo	Average (ms)	Min (ms)	Max (ms)
Use Total in a for loop	0.1	0	4
Use Total with java8 forEach	0.1	0	40
Use Total and java8 map()	0.1	0	6
Use Java8 map and reduce	0	0	2
Use Java8 map, reduce and accumulator	0	0	2
Use Java8 and home-made Collector	0.1	0	6
Use Java8 and ObjectLabKit Calculator	0	0	2
Use Java8 and Parallel Stream	0.1	0	10

First of all, the value generated is the same for every algo, so no bug there it seems.

The results are quite similar except for the Max value, implying a greater deviation in the results. I've used JAmon for measuring min/max and average time.

Surprisingly, it seems that forEach has at least 1 execution at 40ms, which is way above the rest. Otherwise using the ObjectLabKit Calculator seems a good compromise between having to write the reduce correctly (! watch out if the BigDecimal on the right is null!) and using the raw map/reduce. 

The Parallel Stream is not as efficient, as it takes some time to coordinate the tasks and split the list. let's see if it gets any different with more data. 

On my 2012 MacBook Pro (QuadCore) for a list of 50,000 Test instances and the parallelStream is then becoming the most efficient.

Algo	Average (ms)	Min (ms)	Max (ms)
Use Total in a for loop	1	0	20
Use Total with java8 forEach	1.1	0	48
Use Total and java8 map()	2.1	1	40
Use Java8 map and reduce	1.2	1	9
Use Java8 map, reduce and accumulator	1.2	1	10
Use Java8 and home-made Collector	1.4	1	12
Use Java8 and ObjectLabKit Calculator	1.2	1	11
Use Java8 and Parallel Stream	0.6	0	17

So it looks like, when using single thread, that the RAW use of stream.map and reduce is the most efficient but one has to remember how to write it:

  final BigDecimal reduce = list.stream()

        .map(Test::getValue)

        .reduce(BigDecimal.ZERO, 

                (a, b) -> b != null ? a.add(b) : a);

Using the parallelStream (when suitable) reduces the average to 0.5ms but the max is 18ms

  final BigDecimal reduce = list.parallelStream()

        .map(Test::getValue)

        .reduce(BigDecimal.ZERO, 

                (a, b) -> b != null ? a.add(b) : a);

Full code available here at GitHub Gist

ObjectLab Kit 1.3.0 released

We are delighted to announce the release of ObjectLab Kit 1.3.0.

See http://objectlabkit.sf.net

The release is available on Maven Central or Under files in SourceForge.

The Source Code lives at GitHub: http://github.com/appendium/objectlabkit

Feel free to fork and contribute!

This release fixes a couple of bugs but also:

1. includes a JDK8 module using java.time.LocalDate; it is the only module requiring JDK8
2. includes the first official release of ObjectLab Utils a small library with

• Some caches with expiring/timeout but unlike Guava or EHCache, the cache can be refreshed in its entirety in one go; this is suitable only if you can hold the entire dataset in memory; on the plus side, you would hit the generator/DB only once.
• Lots of small utilities for BigDecimal, Integer, Boolean and Collection, mainly to deal with nulls.  And if you deal with BigDecimal, Total, Average and WeightedAverage classes will be very useful.
• ConsoleMenu a way to create user menus for a console/command line application.

Enjoy!

Benoit & the team.

FlatPack 3.4.0 released

Dear All

It is with great pleasure to announce that, after a while one must admit, we have released a new version of FlatPack, v 3.4.0

See FlatPack Website for more information.

Or the change log

The libraries are available on Maven Central and still only required JDK 1.5.

Note that the next release will jump directly to Java8 as we will make use of some cool stream functionalities and Autocloseable features (jdk7).

here is an example of what is coming with 4.0:

	package net.sf.flatpack;
	import java.io.StringReader;
	import java.math.BigDecimal;
	import java.util.List;
	import java.util.stream.Collectors;
	import junit.framework.TestCase;
	/**
	* JDK Streaming test.
	* @author Benoit Xhenseval
	*/
	public class StreamingTest extends TestCase {
	private static class Test {
	private String itemName;
	private BigDecimal price;
	public String getItemName() {
	return itemName;
	}
	public void setItemName(String itemName) {
	this.itemName = itemName;
	}
	public BigDecimal getPrice() {
	return price;
	}
	public void setPrice(BigDecimal price) {
	this.price = price;
	}
	}
	public void testContains() {
	final String cols = "item,price,purchaseDate\r\n"//
	+ "MacBook,1890.20,20140523\r\n"//
	+ "Surface3,850.00,20140524\r\n"//
	;
	final Parser p = DefaultParserFactory.newCsvParser(new StringReader(cols));
	List<Test> ds = p.stream() //
	.map(t -> {
	Test r = new Test();
	r.setItemName(t.getString("item"));
	r.setPrice(t.getBigDecimal("price"));
	return r;
	})// Mapping from Record to Test
	.filter(t -> "Surface3".equals(t.getItemName())) // only keep greetings hello2
	.collect(Collectors.toList());
	// test record 1 with Data in file!
	assertEquals("Size", 1, ds.size());
	assertEquals("Item", "Surface3", ds.get(0).getItemName());
	assertTrue("Price", new BigDecimal("850").compareTo(ds.get(0).getPrice()) == 0);
	}
	}

view raw flatpack-stream.java hosted with ❤ by GitHub

Enjoy!