automating calucation of decimal value from scale for xbrl fact

Posted on Sat 28 July 2012 in XBRLLeave a comment

Today in office or currently in office; (both are right...)

i am making python script that can automate calculation of decimal value from scale for various cases of xbrl instance facts.

Here below, i am giving my current output.

(scale, excel value) (decimal, instance value)

(None, '2345') ('INF', '2345')
('3', '439.10') (-1, '439100')
('-4', '1789000') (4, '178.9')
('-3', '0.006') (6, '0.000006')
('0', '3987') (0, '3987')
('0', '0.0660') (4, '0.066')
('2', '100') (-2, '10000')
('2', '100.0') (-1, '10000')
('2', '100.00') (0, '10000')
('3', '100.660') (0, '100660')
('6', '2.5') (-5, '2500000')
('6', '2.50') (-4, '2500000')
('6', '2.500') (-3, '2500000')
('6', '2.540') (-3, '2540000')
('-1', '0.1') (2, '0.01')
('-2', '0.12') (4, '0.0012')
('-3', '0.123') (6, '0.000123')
('9', '10.5') (-8, '10500000000')
('9', '105') (-9, '105000000000')
('0', '0.02') (2, '0.02')
(None, '0.02') ('INF', '0.02')

Here in 1st tuple(bracket); 1st value is the scale i am applying on cell value on excel, and that value is on 2nd position of this tuple.

and in second bracket(tuple), its what the heading of this post.

1st value of second tuple is its decimal value, while 2nd value is the value which i will send in instance document.

Well here i have also tried on negative scales as well.

It will remove the headache for analyst to worry about decimals (or "precision"; what Bowne guys have on their "Bowne tagger") ; he/she has to concern only about the scale value of fact, if it has any scale then apply it, program will intelligently calculate the decimal value, and if no scale is applied then it will put "INF" as a decimal for all numeric itemTypes.

But let me check on live files.
If you find any mistake please feel free to inform me about that.


Dynamic excel mapping for creating XBRL with help of Python/elementTree/xlrd

Posted on Sat 14 July 2012 in XBRL • Tagged with excel, excel macro, python, python xbrl creation, xbrl creatiion, xlrd, xml/xbrlLeave a comment

Yesterday was a great day for me.. .After 3 months of development i got little bit success, i have created xbrl package from excel with help of python language.

I have developed my own mapping language which needs to be defined in excel which recognizes financial facts and their properties and then my python script recognizes it and make xbrl package.

I have used "xlrd" library to deal with excel, "elementTree" to make XBRL with SQLite at the backed db support.

I am able to create instance document, schema document, all linkbase files (presentation, calculation, label, definition) and its up-to xbrl 2.1 specifications also.

Anyway.. i will need help of VBA expert who can make it automate in excel end(I can do this also but it will take time to reach their) and First let me create its documentation and upload it to my cloud so that others can test/make xbrl package from it.


Adding new line and encoding infomation while creating XML/XBRL with ElementTree/Python

Posted on Tue 10 July 2012 in XBRL • Tagged with elementTree, ElementTree/Python, encoding, new line, ProcessingInstruction, python, xbrl, xbrl creationLeave a comment

I am making xbrl files with python ElementTree/python module. But there is a problem while creating an xbrl file. It writes it into a single line, which creates confusion a lot.

So i have tried to modified the native ElementTree class which writes a xml  part. In windows it is located at the "C:\Python27\Lib\xml\etree\ElementTree.py" . In this file there is a function named as "_serialize_xml";  and i have hacked it a little bit. I have added a "\n" where angle bracket of xml element is defined. Screenshot for that is given below._serialize_xml

So i got my output xbrl as follows.

xbrl
outputHere you can see that now my python script makes xbrl package as i have wanted from it.
Anyway, rest is i want to add encoding information to it, how?

Below is the solution,

    >>>ins_out=open(instant_filename,'wb',1000)

    >>>ElementTree.ElementTree(top).write(ins_out,encoding="ASCII")
    >>>ins_out.close()

here "instant_filename" is the output filename (example; "htwr-20121231.xml") while  "top" is the root element below which i have created my xml/xbrl file. Just add value of encoding to the instance method "write" of ElementTree.ElementTree


Excel Macro to Prevent Excel from Removing Leading and Trailing Zeros in a single Cell or Selection of Cells

Posted on Sat 23 June 2012 in General • Tagged with excel, leading zero, macro, Trailing zeroLeave a comment

With use of macro given below you can apply ="cell_value" on any selection.

Solution is:

[caption id="attachment_397" align="alignleft" width="460"]macro for
preventing leading and trailling zeros in
excel macro for preventing leading and Trailing zeros in excel[/caption]


XBRL Errors Tracking module 2.0 for my company

Posted on Wed 16 May 2012 in XBRL • Tagged with python-cgi, sqlite, xbrl errorsLeave a comment

I have updated my Errors tracking module. Its now "2.0"

It is Python-CGI working under Apache server. I have developed it in spare time while working with Research and Development team at my company Aptara.

In my previous release it was accepting only instance document every time for all three test. Analyst have to upload instance document again and again. It was time consuming and gives load on network and on server too.

So in i have replace them all with single zip upload. Now analyst have to upload xbrl zip package. Zipped XBRL package is relatively small in size(about 1/10th of extracted content), so it will decrease the load on network.

In back-end i am using SQLite database. SQlite is tiny, server-less database. Few high-level SQL queries are not present in SQLite, but if i require those functionality, then i am using python to solve them (Like List-comprehensions).

Requirement of such tool is that, in my company we using "https://examiner.rrd.com"  to find errors from XBRL package. But few errors that are not tracked by it, but needed to be removed. So analyst has to remove them for higher quality of xbrl package. On an average it took 2-3 hrs to check such errors. But all such errors are make available to analyst  in just matter of seconds with use of my tool.

[gallery link="file" columns="2"]

Here test 4 and test 5 are new. In test 4 we are finding elements whose negated and signage needs to be checked. In test 4 it is showing those elements from instance which are negative and doesn't have negated label as pereferredLabel in presentation linkbase. So as you can see in snap-shot it finds that element's value, its balance from company schema/or from US-GAAP taxonomy (while confirming if is from 2009, 2011 or from 2012 UGT), its preferred label as well as its weight from calculation linkbase.

In test 5, we are interested in checking various period type elements and their value in instance.

  • xbrli:dateItemType
  • us-types:dateStringItemType
  • xbrli:dateTimeItemType
  • xbrli:durationItemType
  • us-types:durationStringItemType
  • xbrli:gDayItemType
  • xbrli:gMonthDayItemType
  • xbrli:gMonthItemType
  • xbrli:gYearItemType
  • xbrli:gYearMonthItemType
  • xbrli:timeItemType

So this is it.

This tool is working in all three production facilities (New Delhi, Trivendrum, Pune).

Monthly statistics:

  • total hits is about 10000-30000
  • 4000-8000 number of files uploaded
  • total size of files after extraction is 8GB+
  • total number of systems in intranet are about 300
  • total Time saving 1000hrs-2000hrs (Assumption: if it saves 2 hrs per file)

In the end; SQLite is great.

After seeing it working under such large work load, it creates a great respect for this tiny database. That's why, i want to tell: "If i am not using MongoDB then i am sure, i will using SQLite"