Docufy: Document Generation for APEX made easy!

APEX is well known for his RAD solution and the no-extra cost factor: APEX makes it possible to build on time and within budget your database application for the web. But an application is more then building pages for CRUD operations. Very often you also need to generate well looking documents. And this is standard not possible with APEX. A pity!  Within the Oracle world, Bi Publisher is by far the best solution. But from the pricing point of view, it does not really fit in the ‘cheap’ philosophy of APEX. Further, you can also look for Open Source solutions like Jasper Reports and Birt. Both have a community version that you may use for free.

Nevertheless, we decided to build our own tool. During the presentation at APEX World of last week, organized by OgH, we explained how we came to this decision. On different projects at the Flemish Government we always had the business requirement to generate output for MS-Word (.docx). We first tried to solve this requirement with Jasper Reports. This was fine for simple letters. But the end-user also wanted to change some of those documents after generation … and that was not so easy to do with the output created by Jasper.

So the idea for an own “document generator for Oracle” was born. The main objective was trying to bring some of the better features of other tools together in one solution:

  • Separation of data retrieval and document layout.  The layout is template driven: your template is a MS Word document (.docx). This leaves the possibility open to involve business users in the design and build of the desired output.
  • Simple, easy and light: we want the solution to be as much as possible in the database, in PL/SQL, with no extra software on a middle-tier
  • Metadata driven and nice user interface to manage those data
  • Integration in your application via a PL/SQL API. This application may be developed in the technology of your choice. We use it initially in APEX projects, but it can be integrated in any tool that can do a PL/SQL call (Oracle Forms, any java solution, Formspider, …)

Our custom solution evolved well: we already used it successfully in different projects. Therefor, we decided to make it available for the world and packaged the solution in a tool with the name Docufy. You can find more information on the Docufy-site or you may always send a mail to info@docufy.be.

With Docufy,
we proceed on the APEX philosophy,
there where APEX it self stops:
easy and quick generation of operational documents.

Seminar: Oracle and Reporting(Mechelen 08/11/2012)

Do you have one or more applications on top of on Oracle Database?

Do you want to show this data in documents and/or reports?

Did you know you don’t have to make a big investment in a BI solution to create attractive letters, orders, invoices or lists?

But which reporting tool, from a long list, is the best solution?

After many years of using and testing different tools, we will show and share our experiences of our preferred reporting tools:

  • Oracle Application Express
  • Oracle Reports
  • Oracle BI Publisher
  • JasperReports
  • SQL Word
  • Eclipse Birt
  • PL/PDF
  • DocuFy

We’re going to demo and compare those tools so you can learn the possibilities, advantages/disadvantages, …
Of course we’re also taking the learning curves and prices in count, so you can find out the best solution for you and your company.

More info and Registration(Dutch)

BI Publisher caching results problem

If you are experiencing behavior that indicates that your report is being cached and you can’t refresh it then it could be that a reporting setting is indicating that it can cache your result set.

To check this setting go to “Edit Report” and look at the bottom in the list of options. You will find a check-box with the label “Enable document cache”. Unchecking this check-box will probably solve your problem.

By default, all Oracle Enterprise Repository reports uses Document Caching to reduce database roundtrips on the generation of report data. While the data is cached, the rendered report may not always show the most recent activities in Oracle Enterprise Repository. Setting the cache expiration changes depending on how frequently BI Publisher needs to refresh the data for the second and subsequent rendering of a report.

It is possible to change the default cache expiration (30 minutes) under Admin -> Server configuration -> Cache Section.

More information can be found in the BI Publisher.

BI Publisher showing TOPLINK-7001 error on Scheduled Reports

When you have the next error in BI Publisher Scheduled Reports:


Exception [TOPLINK-7001] oracle.toplink.exceptions.ValidationExceptionException Description: You must login to the ServerSession before acquiring ClientSessions.

You might want to check if the tablespace containing the BI Publisher schema has enough space. There is no procedure to physically remove scheduled reports once they have been run. You can manually delete records from the database (bipub schema) but don’t forget to reclaim the lob size. If you delete records containing LOB’s the size doesn’t automatically free up.

A way to do this manually by using the next command:
ALTER TABLE “table” MODIFY LOB (“column”) (SHRINK SPACE);
* command available from 10gR2

Delete history from the table ‘XMLP_SCHED_OUTPUT’ and then run the above command for all the lob columns: “XML_DATA”, “MESSAGE” ,”STATUS_DETAIL” and “DOCUMENT_DATA” .

Once you have done that, restart BI Publisher using Enterprise Manager and the error will be gone.

How Do You start Building up your DataWarehouse & Reporting Environment (Mark Rittman, New Orleans)

The final day of New Orleans 2 different in-depth sessions were organized: ‘Oracle Tools-Application Expres/ADF’ and the other one ‘Business Intelligence’.

I decided to go to Mark Rittman’s session regarding ‘Advanced Development Techniques Using Oracle BI Suite Enterprise Edition Plus’. It was the first time I had the opportunity to follow a presentation of Mark Rittman and as you could probably figure out yourself, it was very interesting.

So let’s start diging into Oracle BI Suite Enterprise Edition Plus … I will give you all the Tips & Tricks (T) and Watch-Out’s (WO) Mark has shared with us.

1. Use BI Administrator to define the logical, physical and presentation layer:

  • T1: Denormalize dimensional tables into 1 single logical table
  • T2: Data needs to be as close to a star schema as possible
  • T3: Best Approach: Define logical layer first to define labels and names correctly and then define the map for the physical layer
  • T4: Use ‘Direct Query’ in ‘Oracle Answers’ to load Discoverer Workbooks
  • T5: Define data-sources at the DWH level to tidy up stuff. You can add functionality, multiple data-sources to report on (Excel & Oracle) and afterwards level these down into your DWH to have 1 single source of thruth.
  • T6: Use lgo files on BI Server to have a look at the actual queries being fired. You can define different kinds of log levels depending on your needs at a specific point in time.
  • T7: Time-Series Calculations can be used for reporting on historical data but within OBI ther’s limited functionality. If you need this functionality it’s better to use EssBase.
  • WO: Errors and Compilation information isn’t well documented, you have a reference guide but no getting started guide. You can have a look at Mark Rittman’s blog and our iAdvise website if you need course material or information regarding walk-throughs, getting started guides.

2. Use EssBase to speed-up your OBIEE Reporting:

  • T1: Using cubes can speed up data load and querying. Use OBIEE for front end purposes
  • T2: In the next release you can load EssBase Cubes inside your model (in one of the futher releases ;o) )
  • T3: Use ‘Analysis Services’ for the same purposes as EssBase, it’s cheaper ;o)
  • T4: OBIEE uses the aliases defined in EssBase on the dimensional attributes
  • T5: Real Benefit: uses all dimensional metadata defined in EssBase, Analysis Services, … In other words you can have a hybrid OLAP solution integrated in OBIEE
  • T6: JDeveloper allows you to access EssBase and get data from different sources

3. Presentation Server:

  • T1: You can define conditional formatting on columns, e.g. use images available in ‘Oracle Answers’ (Click on the ‘image button)
  • T2: Filter on data using ‘Dashboard Prompt’ defined in the Catalog-tab in ‘Oracle Answers’
  • T3: You can define a Dashboard Prompt on Report Level or Page Level
  • T4: Use hidden dashboards for guided information using ‘Guided Navigation Link’
  • T5: Use Word Add-In to define BI Publisher Reports, best use an older version than Word 2007
  • T6: Use existing ‘Oracle Answer Reports’ or Discovere query and base the report on this data, using Word Add-In
  • T7: ‘Presentation Variable’ in Dashboards needs to match the parameter name being used in BI Publisher to parameterize and filter on report data

These are all the tips & tricks Mark has shared with use, using his experience and knowledge regarding Business Intelligence. It was a great session and hopefully you will have the ability to attend a session of Mark in the future as well.

Tips & Tricks in BIP

I have worked with BIP for a few months and I discover some tips & tricks around developing reports. If you develop too fast or you are unattended to some stuff, you would see some of following errors. I will give you a list where you have to be aware of.

Tips & Tricks 1:

If you create a new datasource and you type a new SQL-query, ended with a semicolon.

For example: SELECT * FROM dual ;

You will get an error message:


When you click on ‘Error Detail’, it returns: ORA-00911: invalid character.

Solution for this problem is that you remove your semicolon at the end and render your report again.

Tips & Tricks 2:

If you create a new template name without uploading a RTF-template first. This means that there is no RTF-template available under ‘layouts’. After you created a template name, you will upload your RTF-template into BIP.

In the picture below, you can see that drop-down box for ‘template’ is empty.

Overview Report:


You go back to tab ‘Layouts’ and you upload your RTF-template. Without returning to template name, you run your report and you will get following error.

Error message:

Error detail: Attribute missing in : ‘url’

Solution, you have to go back to edit your report. Go to your template name and assign your RTF-template that you have uploaded into BIP.

The reason why he gives this error is simple. You create a new template name but you didn’t assign a RTF-template to it. Because this was uploaded after you create a new template name. BIP will not automatically assign new RTF-template to your template name. That’s why you see this error. You have to do it manually.

Unless, if you first upload your RTF-template and then create a new template name. Now, he will take first RTF-template in the list and assign it to your template name. You change it if you want.

Next two tips & tricks is dependent on which version of BIP you’re working on. It seems that the problems in 10.1.3.3 were solved or better, they have a workaround to avoid these problems.

Tips & Tricks 3:

This first problem I met, is that you always need to select a ‘default data source’ if you create a LOV (List Of Values). So I will explain the situation.

When you create a parameter in BIP, it is also possible to attach an LOV to this item. The LOV can be used to choose values dependent on tables in your schema. So if you create a new LOV, you have to choose explicitly a connection data source like ‘HR’ or something else you defined by JDBC Connection in the Admin tab.

You may not select ‘Default Data Source’ as connection because this will render following error:

Error detail: Parameter name: p_dual Can not establish database connection(null)

The reason why you would see this message is because the LOV will be build before there was a default data source connection. So you have to define explicitly your data source connection in 10.1.3.2.

I noticed that the problem is fixed in 10.1.3.3 and you don’t have to select explicitly your connection but you can select it as ‘Default Data Source’.

Tips & Tricks 4:

This topic will handle about more data sources concatenated. If you create more than one data source and you want them concatenated with unique row names.

A short explanation of what I have done. I created two data sources:

  • Name of 1ste datasource = DS_1
  • Name of 2nd datasource = DS 2

Note: In second data source is no ‘underscore’ presented but a space!

I changed the option to ‘Concatenated SQL Data Source’ as Main Data Set on the Data Model Tab. And checked ‘Make row names unique’.

When you run the report, You will get an error message like this:

Error detail: ‘=’ missing in attribute.

The problem is that it is not allowed to use a space in the data source name. So you can change this one or you can obtain to uncheck ‘Make row names unique’. I always choose for the first solution. Also to avoid problems later by the creation of my RTF-template.

Note: If you use a font that is not recognized on the server (where BIP is installed), BIP will use the default font type of Helvetica.

Cross Tab in BIP

The columns of a cross-tab report are data dependent. At design-time you do not know how many columns will be reported, or what the appropriate column headings will be. Moreover, if the columns should break onto a second page, you need to be able to define the row label columns to repeat onto subsequent pages.

You can use the built-in function of Oracle BI Word Plugin to create an Cross Tab. If you want more or less functionality, you have to edit your existing cross tab. So you have to know what every tag means or how to realize.. The following example shows how to design a simple cross-tab report.

This example uses the following XML sample:


&ltROWSET&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtAdministration&lt/DEPARTMENT&gt
&ltYEAR&gt2008&lt/YEAR&gt
&ltQUARTER&gtQ1&lt/QUARTER&gt
&ltSALES&gt3000&lt/SALES&gt
&lt/RESULTS&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtAdministration&lt/DEPARTMENT&gt
&ltYEAR&gt2008&lt/YEAR&gt
&ltQUARTER&gtQ2&lt/QUARTER&gt
&ltSALES&gt2400&lt/SALES&gt
&lt/RESULTS&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtAdministration&lt/DEPARTMENT&gt
&ltYEAR&gt2007&lt/YEAR&gt
&ltQUARTER&gtQ1&lt/QUARTER&gt
&ltSALES&gt1200&lt/SALES&gt
&lt/RESULTS&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtAdministration&lt/DEPARTMENT&gt
&ltYEAR&gt2007&lt/YEAR&gt
&ltQUARTER&gtQ2&lt/QUARTER&gt
&ltSALES&gt3400&lt/SALES&gt
&lt/RESULTS&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtAdministration&lt/DEPARTMENT&gt
&ltYEAR&gt2006&lt/YEAR&gt

&lt/RESULTS&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtHuman Resources&lt/DEPARTMENT&gt

&lt/RESULTS&gt
&ltRESULTS&gt
&ltDEPARTMENT&gtIT&lt/DEPARTMENT&gt

&lt/RESULTS&gt
&lt/ROWSET&gt

From this XML we will generate a report that shows each department and totals the sales by year as shown in the following figure:


The template to generate this report is shown in the following figure. The form field entries are shown in the subsequent table.


The form fields in the template have the following values:


Note: only the first row uses the @column context to determine the number of columns for the table. All remaining rows need to use the @cell context to create the table cells for the column.

Additional Information:

Defining Columns to Repeat Across Pages:
If your table columns expand horizontally across more than one page, you can define how many row heading columns you want to repeat on every page. Use the following syntax to specify the number of columns to repeat:

&lt?horizontal-break-table:number?&gt

where number is the number of columns (starting from the left) to repeat.

Note that this functionality is supported for PDF output only..

Regrouping the XML Data:
The RTF template supports the XSL 2.0 for-each-group standard that allows you to regroup XML data into hierarchies that are not present in the original data. With this feature, your template does not have to follow the hierarchy of the source XML file. You are therefore no longer limited by the structure of your data source.

Using the Context Commands:
To support this requirement, BI Publisher provides a set of context commands that allow you to define the context (or placement) of the processing instructions. For example, using context commands, you can:

  • Specify an if statement in a table to refer to a cell, a row, a column or the whole table.
  • Specify a for-each loop to repeat either the current data or the complete section (to create new headers and footers and restart the page numbering).
  • Define a variable in the current loop or at the beginning of the document.

BI Publisher supports the following context types:

Checkbox in BIP

Since BI Publisher is integrated with MS Office, you can use some features of Word to add extra functionality into your reports. This topic will handle about the usage of checkboxes in your RTF-template. An example below..

We will create an checkbox which status is dependent on a value in the xml. First, we have to create a SQL-query to get our value. Next step is to design a template to add our new functionality and as finishing touch, we upload it to BI Publisher and test it out.

SQL-query:

SELECT 1 as checkbox_ind FROM dual

XML-file:


&lt?xml version="1.0" encoding="UTF-8" ?&gt
&ltROWSET&gt
&ltROW&gt
&ltCHECKBOX_IND&gt1&lt/CHECKBOX_IND&gt
&lt/ROW&gt
&lt/ROWSET&gt

RTF-template:
To add an form element into our template, we have to do first some steps to get our element.

Open MS Word 2007 and go to ‘Word Options’ under the Office-logo. There appears a new window where you can set the option for a developer tab.


Check box ‘Show Developer tab in the Ribbon’. Click ‘OK’ and you will see a new tab into your Word-page.


Follow next step to insert a new form element, in our case a checkbox:


The selected form element appears on your page. We are almost there, we have to add the BI Publisher functionality to get our checkbox work.

Select your checkbox and right click on the icon. You see now the properties of this element. You can set the default value of the checkbox. Choose ‘Add Help Text’ and specify your condition on this element.


Extra information about condition: &lt?CHECKBOX_IND=1?&gt
This consists of two element. First element, CHECKBOX_IND, is xml-tag in your xml-file. Second value is the comparising value that has to be met when returning true. If everything is done, click in both dialog boxes on ‘OK’.

At this moment, we have to prepare BI Publisher to understand our checkbox functionality. We have to add a new font. Log in to BI Publisher and go to ‘Admin’ tab.


Choose ‘Font Mappings’ by ‘Runtime Configuration’. Then ‘Add Font Mapping’.


Properties for this font:


Note: If ‘Arial Unicode MS’ –font is not available on the server. You have first to upload this into directory ../jdk/jre/lib/fonts. Restart your OC4J! Now you can select your font-type.

Last step before we upload our template is configure our reports to use this font. Go back to Admin tab and choose ‘Runtime Configuration’ -> Properties. Edit following setting under RTF-template.


Characters used for checkbox: Arial Unicode MS;9746;9744.

Note: Value 9746 stands for checked checkbox and value 9744 stands for unchecked checkbox.

All configuration is done, the final step is to upload our template into BI Publisher and run our report. You see that your checkbox is checked or unchecked dependent on your value of xml-tag ‘CHECKBOX_IND’.

Additional:
If you use the xml-tag &lt?CHECKBOX_IND=1?&gt in your template, it would return the value true or false. This can also be used if needed.

Practical examples: