Sunday, 29 July 2018

A Human Readable Predicate with ObjectLab Kit

Recently, I have been working with Predicates.

I'm modelling some stuff for a bank but I needed to present updates and changes to the handwritten rules regularly. I don't know about you but wouldn't be nice if we could print the Predicate and not getting something like:

net.objectlab.kit.util.excel.ExcelWorkbookTest$$Lambda$1/868693306@13221655

Therefore, here comes the PrintablePredicate class in the next release of ObjectLab Kit.

See PrintablePredicate.java

The idea is to create an implementation of a Predicate and allow you to give it a name and the values it is comparing against.  We also support AND, OR and NEGATE as per a Predicate.

Let's imagine a small model, a financial Instrument 'Asset' and some basic Predicates.  Imagine that we create a predicate that detects instruments that are either Bonds or Commodities but that they should also be Active.

Using PrintablePredicate, I can combine 2 predicates and when I print the predicate (in an Excel spreadsheet that I generate automatically, more on that later) I can see a nice string "AssetClass in (Bond, Commodities) AND Active".  Here is the code for it:



This tiny class will come with ObjectLab Kit 1.4.1. Enjoy!

Sunday, 22 July 2018

Flatpack 4.0.2 released! 50% speed improvement

Dear All

It is with great pleasure to announce that, after a while one must admit, we have released a new version of FlatPack, v 4.02.

See FlatPack Website for more information.

The libraries are available on Maven Central and still only required JDK 1.8.

The release jumped directly to Java8 as we make use of some cool stream functionalities and Autocloseable features.  We have also spent a little bit of time on performance and improved the overall parsing results by about 50%! 

Enjoy!


Monday, 26 May 2014

How to efficiently add BigDecimals

Anyone who deals with monetary values knows that double/float won't cut the mustard and if you deal with prices and FX rates, then BigDecimal is the only real option.

This comes with a lot of potential issues, BigDecimal methods do not handle null very well (i.e. not at all) and sometimes a bug crops up because BigDecimal returns new instances.


So the ObjectLabKit Util package will help, but here is a question for you... what is an efficient way to sum a list of BigDecimal coming from a Class.

Assume that we have a list of 500 Test instances and that we need to sum the Test.value and that value could be null.

We shall run the test 1,000 times.


Option 1: Use Total in a for loop

Option 2: Use Total with java8 forEach

Option 3: Use Total and java8 map()

Option 4: Use Java8 map and reduce

Option 5: Use Java8 map, reduce and accumulator

Option 6: Use Java8 and home-made Collector

Option 7: Use Java8 and ObjectLabKit Calculator

Option 8: Use Java8 and Parallel Stream



So what are the results?

On my 2012 MacBook Pro for a list of 500 Test instances.
AlgoAverage (ms)Min (ms)Max (ms)
Use Total in a for loop0.104
Use Total with java8 forEach0.1040
Use Total and java8 map()0.106
Use Java8 map and reduce002
Use Java8 map, reduce and accumulator002
Use Java8 and home-made Collector0.106
Use Java8 and ObjectLabKit Calculator002
Use Java8 and Parallel Stream0.1010


First of all, the value generated is the same for every algo, so no bug there it seems.

The results are quite similar except for the Max value, implying a greater deviation in the results. I've used JAmon for measuring min/max and average time.

Surprisingly, it seems that forEach has at least 1 execution at 40ms, which is way above the rest. Otherwise using the ObjectLabKit Calculator seems a good compromise between having to write the reduce correctly (! watch out if the BigDecimal on the right is null!) and using the raw map/reduce. 

The Parallel Stream is not as efficient, as it takes some time to coordinate the tasks and split the list. let's see if it gets any different with more data. 

On my 2012 MacBook Pro (QuadCore) for a list of 50,000 Test instances and the parallelStream is then becoming the most efficient.

AlgoAverage (ms)Min (ms)Max (ms)
Use Total in a for loop1020
Use Total with java8 forEach1.1048
Use Total and java8 map()2.1140
Use Java8 map and reduce1.219
Use Java8 map, reduce and accumulator1.2110
Use Java8 and home-made Collector1.4112
Use Java8 and ObjectLabKit Calculator1.2111
Use Java8 and Parallel Stream0.6017


So it looks like, when using single thread, that the RAW use of stream.map and reduce is the most efficient but one has to remember how to write it:

  final BigDecimal reduce = list.stream()
        .map(Test::getValue)
        .reduce(BigDecimal.ZERO
                (a, b) -> b != null ? a.add(b) : a);

Using the parallelStream (when suitable) reduces the average to 0.5ms but the max is 18ms
  final BigDecimal reduce = list.parallelStream()
        .map(Test::getValue)
        .reduce(BigDecimal.ZERO
                (ab) -> b != null ? a.add(b) : a);


Full code available here at GitHub Gist

ObjectLab Kit 1.3.0 released

We are delighted to announce the release of ObjectLab Kit 1.3.0.

See http://objectlabkit.sf.net

The release is available on Maven Central or Under files in SourceForge.

The Source Code lives at GitHub: http://github.com/appendium/objectlabkit

Feel free to fork and contribute!

This release fixes a couple of bugs but also:

  1. includes a JDK8 module using java.time.LocalDate; it is the only module requiring JDK8
  2. includes the first official release of ObjectLab Utils a small library with
  • Some caches with expiring/timeout but unlike Guava or EHCache, the cache can be refreshed in its entirety in one go; this is suitable only if you can hold the entire dataset in memory; on the plus side, you would hit the generator/DB only once.
  • Lots of small utilities for BigDecimal, Integer, Boolean and Collection, mainly to deal with nulls.  And if you deal with BigDecimal, Total, Average and WeightedAverage classes will be very useful.
  • ConsoleMenu a way to create user menus for a console/command line application.
Enjoy!

Benoit & the team.

FlatPack 3.4.0 released

Dear All

It is with great pleasure to announce that, after a while one must admit, we have released a new version of FlatPack, v 3.4.0

See FlatPack Website for more information.

Or the change log

The libraries are available on Maven Central and still only required JDK 1.5.

Note that the next release will jump directly to Java8 as we will make use of some cool stream functionalities and Autocloseable features (jdk7).

here is an example of what is coming with 4.0:


Enjoy!

Friday, 12 March 2010

Sonar and BlackDuck.com for ObjectLab Kit.

A big thank you to Simon from SonarSource.org to include ObjectLab Kit in their demo site.

Check this out: http://nemo.sonarqube.org/dashboard/index/250253

But we're going to make it better... reach a higher % of compliance...

Also thanks to ohloh.net

https://www.openhub.net/p/objectlabkit

If you are using it... vote for it!

And, yepee, I am ranked 2,400 or something out of 315,000...

https://www.openhub.net/accounts/benoitx

Whatever that means.

Enjoy!

Thursday, 11 March 2010

FlatPack 3.2.0 is released!

FlatPack on SourceForge: a Java (1.4+) flat file parser that handles CSV, fixed length and custom delimiters. The formats are configured in XML, it is fast and released under Apache license 2.0.

http://flatpack.sf.net


Changes in this version include:

New Features:

o Added a getBigDecimal method on DataSet.

Fixed bugs:

o Fixed SF Bug 1869636. The parameters for the XML Map and data file were reversed in the BuffReaderDelimParser.
o Stopped the fixed width parser from removing leading spaces in a data element. Added the ParserUtils.rTrim() method.
o Added check for duplicate column names when using file header for column names.
o Applied patch from Dirk Olmes to prevent duplicate column names in the XML
mapping. IllegalArgumentException is now thrown if a duplicate column name exists in the map. Thanks Dirk...
o doParse() on DBFixedLengthParser was returning a null and was never getting a DataSet returned

Enjoy!

Saturday, 14 March 2009

StatCVS 0.5.0 Beta is out

A new StatCVS library is available for BETA testing at:

StatCVS retrieves information from a CVS repository and generates various tables and charts describing the project development.

http://statcvs.sf.net/beta/statcvs.jar

Site: http://statcvs.sf.net/beta

Manual:
http://statcvs.sourceforge.net/beta/manual.html

The biggest changes are:
- Charts are now quite configurable: colors, size, copyright text, etc
- a Twitter Integration: link, embedding last tweets via Flash or HTML

have a go!

Enjoy

Benoit

Sunday, 4 May 2008

StatCVS 0.4.0 released!

Hi All,

Quick post to let you know that StatCVS, the new member of the family (but by all means not a baby given that it has been around since 2002!) has a new release version 0.4.0.

The release is available on http://statcvs.sf.net

The changes are described here: http://statcvs.sourceforge.net/changes-report.html

Thanks to everyone who participated by sending patches, suggestions and checking the beta!

Enjoy!

Benoit

Wednesday, 2 April 2008

StatCVS joining the family!

Hi All,

StatCVS will soon join the family of projects. Jason Kealey (of StatSVN fame) and myself have been added as project admin.

Our first goal will be to revitalize the community, go through the patches that have been suggested and consolidate the features between StatSVN and StatCVS.

We've already put a Beta Site together. The jar is also available statcvs.jar.

Amongst the significant changes are:

  1. The RepoMap and LOCChurn reports have been added to StatCVS.
  2. An XML export (-xml) is now available
  3. Any comment starting with http://, https:// or simply www. will create an auto-link in the commit report.
  4. A few patches applied
  5. Eclipse cleanUp and new web site with the usual suspects of QA tools (and QALab of course!)


Well, what are you waiting for? If you use CVS, have a look!

Thanks

Benoit

Monday, 24 March 2008

ObjectLabKit 1.1.0 released - Date Calculators for Business and Finance

We are pleased to announce the ObjectLab Kit 1.1.0 release!

http://objectlabkit.sourceforge.net



Changes in this version include:

New Features:

  • Changed JODA dependency to 1.5
  • Feature Requests item #1832345, make the Tenor Serializable Fixes 1832345. Thanks to Kieron Wilkinson.
  • Added 2 methods on factory to check if a calendar is registered.
  • Added method calculateTenorDates with/without a spot lag to enable calculation of a series of Tenor dates without changing the current business date in the calculator.
  • Added method moveByTenor without a spot lag to allow tenor calculation based on the CURRENT date and not the spot lag.
  • Valid Range via HolidayCalendar. HolidayCalendar should replace the simple Set of dates for holidays. A HolidayCalendar MAY contain an early and late boundary, if the calculation break a boundary, an exception is thrown, if there are no boundaries no exception would be thrown. This would ensure that calculations are not going outside the valid set of holidays. Fixes 1575498. Thanks to Paul Hill.
  • Added a standard Tenor 2D. Fixes 1601540. Thanks to Anthony Whitford.
  • Added new handler type ForwardUnlessNegative: a handler that acts like a Forward handler if the increment is positive otherwise acts like a Backward handler.


Fixed bugs:

  • fix NPE issue if the calendar name is null.
  • Deprecated ACT/UST and END/365 Day Count Conventions, which weren't very common. Also added a link to some documentation.
  • The calculation of Spot date should take into account holidays BETWEEN now and spot (aka moveByBusinessDay). Thanks to David Owen.
  • Spelling mistake in the code, sorry for breaking your code with this release. Fixes 1601542. Thanks to Anthony Whitford.





Issues, bugs, and feature requests for ObjectLab Kit
should be submitted to the following issue tracking system:

http://www.sourceforge.net/tracker/?group_id=175139

Have fun!
-The ObjectLab Kit development team

Monday, 5 November 2007

FlatPack 3.1.1. is released.

FlatPack 3.1.1. is released.

It is a simple bug fix release:

  • [1818818] ClassCastException when accessing header or trailer records.

  • Fixed bug in delimited parse when using Reader for data and map. Parameters were being reversed in the code.

  • [1811210] When parsing multi-line delimited files, blank lines inside the elements were being removed from the result of the parse. Blank lines inside a delimited element were also causing a StringIndexOutOfBoundsException.


Released on maven Repositories:
M1: http://objectlabkit.sf.net/m1-repo
M2: http://objectlabkit.sf.net/m2-repo

Enjoy!

Paul & Benoit.

Wednesday, 10 October 2007

J2EE and Swing Jobs @ ObjectLab London.

Hi All,

Does this qualify as "ObjectLab Open Source News"? may be not exactly... it is simply ObjectLab News, so sorry in advance:

ObjectLab Financial recently launched its global portfolio financing product; we’re in the final phases of rolling out Release 1.0 to our first client and already have several additional leads.

As such, we are recruiting and have 2 positions in London:
• A proficient J2EE Developer: JDK 5, EJB/POJOs, Spring, Hibernate, JBoss, Mule, JMS, ActiveMQ, XML, JAXB etc
see http://www.objectlab.co.uk/jobs/index.shtml?j2ee.inc

• A proficient Swing Developer: JDK 5, Swing, Spring 2 (Spring Rich Client a plus), Jasper Reports, etc
see http://www.objectlab.co.uk/jobs/index.shtml?swing.inc

Both roles should attract dedicated, hardworking developers looking for a challenging and rewarding job opportunity. Working in a small, delivery-focused team, you’ll have the chance to use your skills and knowledge, to find the best solutions to the challenges presented, as well as shaping the future of a new company with a great product!

We will offer a competitive package that will be complemented with performance related bonuses (including stock options).

Feel free pass onto experienced Java Developers that would fit the requirements.

Please use the links to contact us more privately.

Many thanks

Benoit.

Sunday, 30 September 2007

FlatPack 3.1.0 released with Mule Contribution

Second post in 3 days... wow! Things are happening!

We are pleased to announce release 3.1.0 of FlatPack for Java 1.4+.

FlatPack is the new name for PZFileReader as the project has outgrown the initial scope of reading files...

Open Source flat file parser (CSV, Fixed Length, Custom) using XML to configure formats.

http://flatpack.sourceforge.net

This is an important release with a new name and package structure. Users of previous version should find it easy to migrate as most classes have kept their original name.

A major development is the experimental release of writers for exporting DataSets. We would like to thank Dirk and Holger from the Mule Project for their kind contribution to FlatPack. We're looking forward to the result of using FlatPack in Mule, a great Open Source ESB.

This release also adds a few convenience methods on a DataSet and the Parser classes, fixes a couple of bugs.
More on changes at: http://flatpack.sf.net/changes-report.html.

FlatPack is released under the business friendly Apache License v2.0.

The library is small, lightweight and does not force you to adopt a framework.

The implementation is useful to any business that deal with flat files. Not only can it parse very quickly some CSV or any-user defined delimiter, this library can parse FIXED LENGTH files.

The library allow you to define an XML mapping (or in a database) of the format of your file. Once this is done, the parsed data can be accessed via a simple name lookup mechanism.

It is our aim to publish at some point some well know file formats for your immediate use. Please contribute if you have some standard files...

It is available for download via SourceForge or the Maven Central Repository (both Maven 1 and Maven 2). The homepage has some very quick examples.

Maven Repositories:
M1: http://objectlabkit.sf.net/m1-repo
M2: http://objectlabkit.sf.net/m2-repo

ObjectLab is not new to the open-source community having used numerous OS projects, It has recently launched the ObjectLab Kit family, including:
- QALab (http://qalab.sourceforge.net), a tool that keeps track over-time of the static analysis results from FindBugs, Checkstyle, PMD, Cobertura etc.
- DateCalculators (http://objectlabkit.sourceforge.net), a set of generic lightweight and thread-safe Date calculators for Business and Finance.
- JTreeMap, (http://jtreemap.sourceforge.net), probably the only Java Open Source implementation of treemap/heatmaps, available as a Swing or SWT component.
- StatSVN, (http://www.statsvn.org), statistics for your Subversion repo.

We would like to thanks our friends and colleagues for their help, reviews and suggestions.

Sorry for the long post...

Feel free to pass on to people who may be interested.

Enjoy!!

Paul Zepernick and Benoit Xhenseval

Friday, 28 September 2007

ObjectLabKit selected for Open Financial Market Platform

Hi

I meant to send this a long long time ago... So here is the not-so-new news.

My friend Neil Barlett (Mr OSGi and Eclipse) spotted this mention of the ObjectLabKit (DateCalculator) as part of the proposal for the Open Financial Market Platform.

http://www.eclipse.org/proposals/ofmp/

I can only say this: Wow!

I hope it succeeds as the main reason for creating this little library was our frustration at re-inventing the wheel so many times... I know a couple of big investment banks using it now, so it was worth it!

Back to work now...


Benoit

Sunday, 15 July 2007

Accessing JavaBeans Nested Properties: testing Spring, BeanUtils and OGNL

One our application needs to access properties from a javabean using reflection
e.g. get(“property1”, object)…

Rather than re-inventing the wheel, I thought that we should use a library. There are quite a few that do this kind of get/set properties… So the question was: Which One???

I know of:

  1. Spring beans (2.0.5) http://www.springframework.org

  2. Apache Commons BeanUtils (1.7.0) http://jakarta.apache.org/commons/beanutils/

  3. my colleague Gerald mentioned OGNL from www.ognl.org (pronounced ‘like a drunken orthogonal’ to quote their documentation).

So with 3 candidates… which one is the best performing?

OGNL seems to be, by far, the most flexible and rich library, but does that means it runs like a dead dog?

I limited the problem to accessing a property value: being simple, nested or as part of an array.
The Test: I shall access 100,000 a series of 8 properties. The classes are:

public class A {
private int intProperty;
private Long longProperty;
private String stringProperty;
private Date dateProperty;
private B b = new B();
}
public class B {
private int intProperty = 5;
private C c = new C();
private D[] d = new D[10];
}
public class C {
private String stringProperty;
}
public class D {
private int intProperty = 1;
}


The Test creates one instance of A, that contains 1 instance of B which contains 1 instance of C and an array of 10 Ds. I hope this is clear…
The set of properties to get are: "intProperty", "longProperty", "dateProperty", "stringProperty", "b.intProperty", "b.c.stringProperty", "b.d[1].intProperty", "b.d[7].intProperty".

So… the results?

LibraryTotal time (ms)average per set (micro sec)
Spring1,783 ms17.8 micro sec
Bean Utils2,242 ms22.4 micro sec
OGNL50,293 ms503 micro sec
OGNL Expression1,595 ms16 micro sec


What does this tell us?

OGNL is at the same time the slowest and the fastest library on my laptop (Lenovo, dual-core) under java 1.5.0_10. OGNL has 2 mechanisms, one is simply to call Ognl.getValue(“pathToProperty”, object) and the other one is to evaluate the expression upfront by Object expression = Ognl.parseExpression(“pathToProperty”) and then Ognl.getValue(expression, object);

The second one is the fastest mechanism so, if you have the ability to ‘pre-compile’ your expressions, OGNL is for you… otherwise Spring Beans is doing a good job!

The entire source code and Eclipse project is available here, feel free to comment and tell us about your experience.

Enjoy!

Friday, 8 June 2007

Spring prototypes and auto-wire byType are expensive

In designing a new, very performance-sensitive part of our systems we investigated the runtime performance of retrieving beans (singleton and prototype) from a Spring bean factory versus creating them via a hand-coded factory. For the Spring code we also measured any additional overhead of auto-wiring beans and doing dependency checks on beans.

The test repeatedly retrieves 5 beans which are the roots of a highly interconnected object graph (comprising 4 other beans) from a Spring bean factory. In the prototype tests each bean in that graph is a Spring prototype bean, i.e. a new instance is create whenever a bean is needed from the bean factory. In the singleton tests each bean in that graph is a Spring singleton and so the same instance is returned every time a bean is retrieved. By comparison, the hand-coded factory always creates each object in that graph and hence behaves identical to the Spring prototype test.

The results give the time in nanoseconds for retrieving a bean (which is the root of the object graph) from the factory:



















































Bean retrieval / nanosecondsSpring 2.0.5Spring 2.0.4Spring 2.0.3Spring 2.0.2
Spring, prototype, autowire=byType, depend check442365500050945551958805
Spring, prototype, autowire=byName,
depend check
253870255557667352678782
Spring, prototype, no autowire, depend check172412173539624171639640
Spring, prototype, no autowire,no depend check162300162950586144600320
Spring, singleton, autowire=byType, depend check60884110751134
Spring, singleton, autowire=byName,depend check71084010691132
Spring, singleton, no autowire,depend check60984210971161
Spring, singleton, no autowire, no depend check61483711021135
No Spring (hand-coded factory),always create (prototype)284284288292
The numbers speak for themselves, but the shown figures visualise them.



The code we used to perform these measurements and all results are available here.

Wednesday, 30 May 2007

Goodbye PZFileReader! Hello FlatPack!

Paul and I have been muling (pun intended... more on that later!) about the name of our parser project. PZFileReader is very useful and originated in the world of "reading files", its capabilities have outgrown the name for quite some time.

We then thought about the basics, what does this project do?

Well, it parses files, string or messages that are in a delimited format (e.g. csv) or fixed length format (when a field is delimited by an offset and a length). It would even support parsing records that are across multiple lines. Furthermore, the project allowed an XML definition of a given format... Some work is also ongoing to create 'Writers' that would allow you to create such delimited, we hope to reveal more shortly.

And so.. enter FlatPack!

The common denominator of those files/messages is that they are "flat" and not hierarchical a la xml.

We were lucky enough to get the url: http://flatpack.sf.net

The packaging and website will be updated soon!

Enjoy!


Benoit & Paul

Sunday, 29 April 2007

JTreeMap 1.1.0 released!


Hi *,

ObjectLab is very pleased to announce the immediate release of JTreeMap 1.1.0, a heatmap/treemap visual library for JDK 5.0. We believe this is the only open source library of this kind under a business friendly license.

Towards the end of 2006, ObjectLab got involved with JTreeMap as part of a financial application. Laurent Dutheil had been developing it but found it more time consuming that anticipated. It is where ObjectLab offered some expertise and the result is the new release 1.1.0. We acknowledge and thank Laurent for his great contribution.

Home page: http://jtreemap.sourceforge.net

The release is available on SF and on 2 Maven repositories (whilst we’re going through the process of adding it in the official repository). The ObjectLab Open Source repositories are:

http://objectlabkit.sourceforge.net/m2-repo

and

http://objectlabkit.sourceforge.net/m1-repo

The question of how to represent and visualize a lot of information at a glance is a hot topic in IT. A Treemap, also known as Heatmap, is an important tool for this. A TreeMap graphically represents a hierarchical structure.

Typically, the hierarchy will involve a tree of nodes of different sizes and different colours. The size and colours are determined by parameters such as the relative importance of a node in comparison to the full size. A well known examples is the Map of the Market on www.smartmoney.com but their library is not open source (and very pricey!).

JTreeMap comes in 2 flavours: JTreeMap for Swing and KTreeMap for SWT.

Enjoy!

Benoit & the rest of the ObjectLab Team.

Thursday, 19 April 2007

Coming QALab: Emma, XML FindBugs, new XML, new Chart & more!

I thought that it may be interesting to do something that Microsoft has been doing for years, like announcing a coming release of something... The difference is that, this is not vapo(u)rware, it will turn up!

http://qalab.sourceforge.net.

So, what are the items we're working on?

  • Emma support is coming, thanks to the contribution of Robert Crawford.

  • Support for the XML output from Findbugs (rather than the xdoc one), some people had to run findbugs twice due to this limitation.

  • A new Spider Chart will show 3 snapshots of multi-dimensional data at 0, 30 and 90 days. You'll be able to see what is happening at a glance!

  • Automatic migration of the QALab XML to a new format that will also add the notion of "project" and "module" making it more hierarchical... one step closer to aggregation!

  • a bit of code re-org to keep the house tidy!

  • Some experimental work with storing QALab stats in a DB


So... when will it be available? Like cooking, it will be ready when it's ready... but for the impatient type, all these features are in Subversion!

Finally, we're willing to add more tools and are welcoming contributions, especially for Clover! Should not be difficult... Our main problem is that there are only 24 hours in a day!

Until next time,

Benoit