This page will store information about features I'd like to see implemented in OFBiz. Nothing in it is official or already available in OFBiz; chances are that what you see listed here will never be implemented, since they represent my own point of views.
However I think it is a good idea to put these notes in a public site, so that others will have a chance to review them and if interested, add their comments.
Implementing Business Intelligence in OFBiz
NOTE: most of what is described here is implemented in the new "Business Intellicence" component (that is a POC of the ideas expressed here).
We should really consider to add (or integrate) in OFBiz some tools for datawarehousing.
I don't know how Pentaho works, and I've still not researched for similar tools but I think before looking outside we should set up something in OFBiz for this: I'm thinking of a new entity group (e.g. org.ofbiz.datawarehouse) to store denormalized/customized entities that will be the basis of the new reports (these entities could be stored in a different db); triggers or scheduled services could fill the entities; we should design the whole thing so that the layout (and supporting services) of the entities will be easy to customize, and we could include in SVN the basic layout.
I guess that this would allow the creation of (simple) reports (using the widgets?) based on complex analysis and data manipulation (done by the scheduled services/triggers) but also more complex ones using an external reporting tool.
And about simpler reports (but this is a bit off topic here) I strongly believe that we should all use the screen/form widgets and just forget about JasperReport etc... because with them it is already possible to create very quickly simple/nice/easy-to-customize reports (in html/pdf/xml).
While I'm studying the book "The Datawarehouse Toolkit" (OFBiz related books
), I'm trying to draft out a proposal for the implementation of base datawarehousing features in OFBiz (a separate set of entities for dimensions, facts and start schemas; ETL services based on minilang, tools to manage the dimensions tables and synchronization).
My goal is this: once we have a set of star schemas (facts and dimensions) derived from OFBiz entities and based on best practices, and the tools to manage the data in them, we could integrate a visual reporting tool to run reports against them (or just use, with some improvements the form widgets).
It may seem an ambitious plan, but I think that many of the building blocks to complete it are already in the framework, we'll just have to improve and fine tune them.
I'm taking notes about tool extensions that will help to build the datawarehouse; here are a few examples of these very preliminary requirements:
- add support in the entity engine for view-entities that are stored in the db as sql VIEWs (it seems that we will need many different views over the same shared dimensions; for example the "Date" dimension, could play different roles in the datawarehouse)
- implement util methods to populate the Date dimension (that will store all the days of the years of interest for the analysis) and the Time dimenstion (all the minutes of a day)
- a new type of view-entities, used to implement star schemas, where the table joins are only performed if the fields of the referenced entities are actually included in the select list; and all the fields from the referenced entities (the dimensions) should be automatically grouped together/collapsed (if they contain the same values)
- ...
The star schema will be built in a relational database but essentially is composed of:
a fact table (e.g. "Sales Transactions")
a set of dimension tables (e.g. "Products", "Date", "Time", "Stores" etc..)
Usually the fact table is only useful inside one star schema, while dimensions are shared among many star schemas.
I'd like to build a common, based on best practices set of dimensions (and a few fact tables that use them) derived from the OFBiz data model.
You can then use the tool you want to run your reports/analysis etc...
However I'd like also to provide some simple tools, based on form widgets to create some simple reports based on the star schemas above but mainly as a PoC.
This is the first draft of the layout of the entity packages for the olap data model:
-- in the "common" component
org.ofbiz.olap
org.ofbiz.olap.staging
org.ofbiz.olap.common.dimensions (e.g. "Date", "Time" etc...)
-- application specific entities (e.g. in the "product" component)
org.ofbiz.olap.product.dimensions (e.g. "Products", "Stores" etc...)
org.ofbiz.olap.product.facts (e.g. "Sales Transactions" etc...)
org.ofbiz.olap.product.starschemas (e.g. "Sales Order Process" a view composed by one fact and many dimensions)
etc...
Note from David Jones:
there are some interesting generic star schemas in the Data Model Resource Book. There is a sales one with some requirements in the form of questions on page 370 of volume 1, and a model diagram on page 371.
The data warehouse entities should go into their own entity group, a new group, with it's own datasource in the OOTB entityengine.xml file. A good group name might be something like org.ofbiz.olap (as opposed to oltp which characterizes most of the current entities.
Alternative Naming convention:
org.ofbiz.olap.staging
org.ofbiz.olap.dimensions.* (common, product, order etc... each group defined in its own component)
org.ofbiz.olap.facts.* (product, order etc... each group defined in its own component)
org.ofbiz.olap.starschemas.* (product, order etc... each group defined in its own component)
Enhancing list based form widgets to better serve as a simple and easy to use reporting tool
I really think it's time to add a few more features to the list based form widgets.
As you probably know, form widget definitions, if used inside screenfop screens, are rendered as xsl-fo (and then pdf) templates.
This, in my opinion, makes the widgets a great tool for simple (list based) reports (both html and pdf).
There is one feature that is missing and I'd love to see implemented: the ability, in list-based form definitions, to define special rows for totals/subtotals and for row group breaks.
NOTE: as of revision 550168 most of what is described here is possible.
For example:
let's say we have a list like this:
orderId | orderItemSeqId | amount
---------------------------------
WS10000 | 0001 | 55.00
WS10000 | 0002 | 45.00
WS10001 | 0001 | 10.00
WS10001 | 0002 | 20.00
it would be great to have a way to render it in this way:
orderItemSeqId | amount
---------------------------------
orderId: WS10000
0001 | 55.00
0002 | 45.00
total for order: 100.00
-----------------------
orderId: WS10001
0001 | 10.00
0002 | 20.00
total for order: 30.00
-----------------------
grand total: 130.00