Monday, 26 May 2014

How to efficiently add BigDecimals

Anyone who deals with monetary values knows that double/float won't cut the mustard and if you deal with prices and FX rates, then BigDecimal is the only real option.

This comes with a lot of potential issues, BigDecimal methods do not handle null very well (i.e. not at all) and sometimes a bug crops up because BigDecimal returns new instances.
BigDecimal bd1 = new BigDecimal(10);
bd1.add(new BigDecimal(20)); // oops
// bd1 is still 10
// and the result of the addition is... lost
// USING Total From ObjectLabKit
Total total = new Total(new BigDecimal(10));
total.add(new BigDecimal(20));
// total is now 30
System.out.println(total.getTotal());
view raw bug.java hosted with ❤ by GitHub


So the ObjectLabKit Util package will help, but here is a question for you... what is an efficient way to sum a list of BigDecimal coming from a Class.
private static class Test {
private final BigDecimal value;
public Test() {
value = null;
}
public Test(final int value) {
super();
this.value = BigDecimal.valueOf(value);
}
public BigDecimal getValue() {
return value;
}
}
view raw Test.java hosted with ❤ by GitHub

Assume that we have a list of 500 Test instances and that we need to sum the Test.value and that value could be null.

We shall run the test 1,000 times.


Option 1: Use Total in a for loop

private static void useTotal(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+Loop");
final Total total = new Total();
for (final Test t : list) {
total.add(t.getValue());
}
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", total.getTotal(), mon.getLabel(), mon.getLastValue(),
mon.getAvg(), mon.getMin(), mon.getMax());
}

Option 2: Use Total with java8 forEach

private static void useTotalAndForEach(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+forEach");
final Total total = new Total();
list.stream().forEach(t -> total.add(t.getValue()));
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", total.getTotal(), mon.getLabel(), mon.getLastValue(),
mon.getAvg(), mon.getMin(), mon.getMax());
}

Option 3: Use Total and java8 map()

private static void useTotalAndMap(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+Map");
final Total total = new Total();
final Iterator<BigDecimal> it = list.stream().map(Test::getValue).iterator();
while (it.hasNext()) {
total.add(it.next());
}
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", total.getTotal(), mon.getLabel(), mon.getLastValue(),
mon.getAvg(), mon.getMin(), mon.getMax());
}

Option 4: Use Java8 map and reduce

private static void useMapAndReduce(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+reduce1");
final BigDecimal reduce = list.stream().map(Test::getValue).reduce(BigDecimal.ZERO, (a, b) -> b != null ? a.add(b) : a);
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
mon.getMin(), mon.getMax());
}

Option 5: Use Java8 map, reduce and accumulator

private static void useReduceAndAccumulator(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+reduce2");
final BigDecimal reduce = list.stream().map(Test::getValue).reduce(BigDecimal.ZERO, BigDecimalAccumulator.INSTANCE);
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f count %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
mon.getMin(), mon.getMax());
}

Option 6: Use Java8 and home-made Collector

private static void useCollector(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+collect");
final BigDecimal reduce = list.stream().map(Test::getValue).collect(new ToTotalCollector()).getTotal();
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
mon.getMin(), mon.getMax());
}
public static class ToTotalCollector implements Collector<BigDecimal, Total, Total> {
@Override
public Supplier<Total> supplier() {
return Total::new;
}
@Override
public BiConsumer<Total, BigDecimal> accumulator() {
return Total::add;
}
@Override
public Function<Total, Total> finisher() {
return Function.identity();
}
@Override
public BinaryOperator<Total> combiner() {
return (t1, t2) -> {
t1.add(t2);
return t1;
};
}
@Override
public Set<Characteristics> characteristics() {
return Collections.unmodifiableSet(EnumSet.of(IDENTITY_FINISH, CONCURRENT));
}
}

Option 7: Use Java8 and ObjectLabKit Calculator

private static void useCalculator(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+calc");
final BigDecimal reduce = Calculator.sum(list, Test::getValue);
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
mon.getMin(), mon.getMax());
}
private static class Calculator {
public static <T> BigDecimal sum(final Collection<T> collection, final Function<T, BigDecimal> mapper) {
return collection.stream().map(mapper).reduce(BigDecimal.ZERO, (l, r) -> r != null ? l.add(r) : l);
}
}

Option 8: Use Java8 and Parallel Stream

private static void useParallelCollector(final List<Test> list) {
final Monitor mon = MonitorFactory.start("Total+collecP");
final BigDecimal reduce = list.parallelStream().map(Test::getValue).collect(new ToTotalCollector()).getTotal();
mon.stop();
System.out.printf("%10.0f in %13s %5.1f Avg:%5.1f Min:%5.1f Max:%5.1f %n", reduce, mon.getLabel(), mon.getLastValue(), mon.getAvg(),
mon.getMin(), mon.getMax());
}


So what are the results?

On my 2012 MacBook Pro for a list of 500 Test instances.
AlgoAverage (ms)Min (ms)Max (ms)
Use Total in a for loop0.104
Use Total with java8 forEach0.1040
Use Total and java8 map()0.106
Use Java8 map and reduce002
Use Java8 map, reduce and accumulator002
Use Java8 and home-made Collector0.106
Use Java8 and ObjectLabKit Calculator002
Use Java8 and Parallel Stream0.1010


First of all, the value generated is the same for every algo, so no bug there it seems.

The results are quite similar except for the Max value, implying a greater deviation in the results. I've used JAmon for measuring min/max and average time.

Surprisingly, it seems that forEach has at least 1 execution at 40ms, which is way above the rest. Otherwise using the ObjectLabKit Calculator seems a good compromise between having to write the reduce correctly (! watch out if the BigDecimal on the right is null!) and using the raw map/reduce. 

The Parallel Stream is not as efficient, as it takes some time to coordinate the tasks and split the list. let's see if it gets any different with more data. 

On my 2012 MacBook Pro (QuadCore) for a list of 50,000 Test instances and the parallelStream is then becoming the most efficient.

AlgoAverage (ms)Min (ms)Max (ms)
Use Total in a for loop1020
Use Total with java8 forEach1.1048
Use Total and java8 map()2.1140
Use Java8 map and reduce1.219
Use Java8 map, reduce and accumulator1.2110
Use Java8 and home-made Collector1.4112
Use Java8 and ObjectLabKit Calculator1.2111
Use Java8 and Parallel Stream0.6017


So it looks like, when using single thread, that the RAW use of stream.map and reduce is the most efficient but one has to remember how to write it:

  final BigDecimal reduce = list.stream()
        .map(Test::getValue)
        .reduce(BigDecimal.ZERO
                (a, b) -> b != null ? a.add(b) : a);

Using the parallelStream (when suitable) reduces the average to 0.5ms but the max is 18ms
  final BigDecimal reduce = list.parallelStream()
        .map(Test::getValue)
        .reduce(BigDecimal.ZERO
                (ab) -> b != null ? a.add(b) : a);


Full code available here at GitHub Gist

No comments :