You are on page 1of 17

Name: Author: Email: Date:

IPD Parse
Mike West

Ver. 2.2.8

rapanui21@yahoo.com
June 2012

Macro Description: The IPD Parse macro is designed to read a Blackberry IPD file and create reports for certain data bases detected on
the IPD file. These reports are written to Excel Worksheets (within the IPD Parse Workbook), a separate Workbook with only the DB report worksheets, a single composite text file, and optionally to some individual text files. IPD Parse runs on Windows and Excel. The input IPD file must be created by the Blackberry Desktop Manager program. The IPD file may be a comprehensive file containing a full backup with all Blackberry DBs present or it may be a customized file created with the Advanced option of the Backup function and contain a single DB. Without a macro of this nature, you would have limited options to view or migrate your Blackberry information from your phone to a non-Blackberry platform. This macro helps you view and use various portions of the otherwise unreadable binary IPD file.

Background:
While I do not provide any contractual technical support, I would be happy to assist you in any manner. I do not charge for my services. All my services are free. Please feel free to email me (rapanui21@yahoo.com). This macro represents hours and hours of research, coding, debugging, testing, and writing documentation. I did this as a labor of love. My aim is to provide you with a valuable tool.

Development Platform:
Following is the hardware/software I used to develop this macro. Windows XP, Excel 2003, Visual Basic 6.3,

Blackberry Curve 8310, Blackberry Desktop Manager V5.0. Binary Viewer; Freeware Version 2.0.9.1214

Processing Platform:
Following is the hardware/software on which this macro has been successfully tested. Windows XP, 2000, Win 7 NOTE: This macro does not run well on Vista. Excel 2003, 2007, 2010 Visual Basic 6.3, Blackberry Curve, Pearl, Torch, Bold Blackberry Desktop Manager V5.x, V6.x

Revision History:
Ver 1.7 (June 2011) Ver 1.8 (August 2011) Ver 2.2 (February 2012) Ver 2.2.4 (March 2012) Ver 2.2.7 (May 2012) Ver 2.2.6 (June 2012)

Getting Started:

There are three important pre-requisites to running the IPD Parse macro.
1.

Input Preparation
A. B. IPD file DST file

2. 3.

Excel Security settings IPD Parse options

1. Input preparation: A. IPD file


This macro requires that the input file be generated from the Blackberry Desktop Manager. Basically, any IPD file may be used as input to the IPD Parse macro. The macro detects which DBs are found on the IPD file and produces reports accordingly.

If you are using BlackBerry Desktop Manager V4.1 to 5.0.1, see Article ID: KB12487 in the BlackBerry

Knowledge Base.

(Google 'BlackBerry KB12487).

If you are using BlackBerry Desktop Manager V6.0, see Article ID: KB23680 in the BlackBerry Knowledge Base. (Google 'BlackBerry KB23680). Note: If performing a Selective Backup, enter'No' when asked if you want 'Encryption'.
The IPD Parse macro provides calculations for Daylight Saving Time (DST), also referred to as Summer Time, for all time zones world-wide (as appropriate).

B. DST file

USA
DST calculations for the USA are based on the appropriate United States Federal law. These calculations are performed automatically and internally by the macro with no actions required by the user. From 1986-2006, DST begins at 2 a.m. on the first Sunday of April and ends at 2 a.m. on the last Sunday of October. From 2007-current, DST begins at 2 a.m. on the second Sunday of March and ends at 2 a.m. on the first Sunday of November. The USA States of Hawaii, Indiana, and Arizona have and/or had DST rules different from the USA Law. The optional override should be utilized, as appropriate, for these locations.

Non-USA
Since there are so many variations to DST rules around the world, IPD Parse provides for an optional override where users can declare the DST dates to be used for their location. Any user whose non-USA location does not observe DST may run the macro without the presence of the optional DST override file. This will cause all DST calculations to be bypassed. Any user whose non-USA location does observe DST should create and make available the optional DST override file. This will cause all DST calculations to be made using those specified rules.

IPD DST.xls override file specifications: File Name:

IPD DST.xls

File Location:
IPD DST.xls must reside in the same directory path as the input IPD file.

Worksheet:
DST data must be in the first worksheet of IPD DST.xls.

File Format:
Column A is the date/time of the Start of the DST period. Column B is the date/time of the End of the DST peroid. The date/time may be entered in any valid Excel date/time format. Successive entries are entered on successive rows. No blanks rows should be entered. No column headings should be entered. Enter a separate row for every year represented on your input data.

Example:
Row #1 Row #2 Row #3 Row #4 Row #5 Row #6 Row #7 Row #8 Col A ---------------------4/30/2006 02:00 AM 3/11/2007 02:00 AM 3/9/2008 02:00 AM 3/8/2009 02:00 AM 3/14/2010 02:00 AM 3/13/2011 02:00 AM 3/11/2012 02:00 AM 3/10/2013 02:00 AM Col B -----------------------10/29/2006 02:00 AM 11/4/2007 02:00 AM 11/2/2008 02:00 AM 11/1/2009 02:00 AM 11/7/2010 02:00 AM 11/6/2011 02:00 AM 11/4/2012 02:00 AM 11/3/2013 02:00 AM

Note: The dates in the above example are actual USA DST start and end dates.

----------------------------------------------------------------------------

2.

Excel Security settings:

Excel, by default, does not allow macros to run. You must change your Excel Security settings to allow IPD Parse (or any macro) to run. Failure to do this will result in a VBA Run Time error 1004.

Excel 2003
1. Select Tools -> Macro -> Security Select the Security Level tab. Select either Medium or Low and Press OK. 2. Select Tools -> Macro -> Security Select the Trusted Publishers tab. Place a checkmark to select Trust access to Visual Basic Project.
Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.

Excel 2007
1. You must save the IPD Parse with an xlsm extension to indicate it is a macro-enabled workbook. Save As, change the type to Excel Macro-Enabled Workbook. 2. Select the main Excel menu (by selecting the Excel logo on the top left of the spreadsheet). At the bottom right of this menu, select Excel Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. From within the Macro Settings option, select Enable all macros How about the 2nd & 3rd option? Also, place a checkmark to select Trust access to the VBA project object model. Then Press OK. Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.

Excel 2010
1. You must save the IPD Parse with an xlsm extension to indicate it is a macro-enabled workbook. Save As, change the type to Excel Macro-Enabled Workbook. 2. Select the File tab, then select Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. From within the Macro Settings option, select Enable all macros . How about the 2nd & 3rd option? Also, place a checkmark to select Trust access to the VBA project object model. Then Press OK. Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.

Trusted Locations (Excel 2007 and 2010 only)


Another easier option for allowing macros to run is to place them in a Trusted Location. If a workbook is in a Trusted Location, Excel will omit the usual macro security checks, thus making it enabled regardless of the macro Security settings. You can do this by placing your workbook which contains a macro in a directory of your choice and then identifying that directory as a Trusted Location. To add a new location to the list of Trusted Locations, do the following:

Excel 2007
Select the main Excel menu (by selecting the Excel logo on the top left of the spreadsheet). At the bottom right of this menu, select Excel Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. Select the Trusted Locations option from the left and menu. Press Add new location and follow the prompts.

Excel 2010
Select the File tab, then select Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. Select the Trusted Locations option from the left and menu. Press Add new location and follow the prompts. Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.

3.

IPD Parse options:


The main screen is displayed when the IPD Parse Excel Workbook is loaded.

Enter the 8 input items as described below:


1. Enter the fully qualified File Name of the input IPD file or click Browse.
The IPD file is created by the Blackberry Desktop Manager as described previously. The file is typically named something like Backup-(2012-02-09).ipd . You would need to prefix the file name with the path. You can make this entry manually or click to browse. This is a required entry.

2. Enter an existing Folder for output files to be written or click browse.


You can make this entry manually or click to browse. This is a required entry.

3. Click on drop down menu to select your Time Zone information.


This information is needed to accurately compute date fields Click on the Time Zone field, then click the drop down menu arrow. Click on your time zone. This is a required entry. Note: This selection also determines how DST will be calculated.

4. Enter Y to have quotes placed around fields on CSV data. Otherwise leave it blank.
This is an optional entry. Some applications require quotes around data fields. Enter Y (or y) if applicable. See CSV Tutorial for details.

5. Click on drop down menu if you want to select a single report to be written to text file. Leave blank to write All Reports.
This is an optional entry. If you want only one report written to the composite text file, select the name of that report. Valid Report names are: Memos, Calendar, Contacts, Phone Call Logs, SMS, PIN, Tasks, and CSV) This option may be used, for example, when you want just the CSV data for future downstream processing.

6. Enter Y to write SMS report to the text file only and not to the xls SMS worksheet. Otherwise leave it blank.
This is an optional entry. This option may be used, for example, when you have (or think you may have) greater than 65,536 SMS messages on your IPD file (which is an Excel row limitation). This would allow you to view your entire SMS report on the text file. Enter Y (or y) if applicable.

7. Enter Y to create a vCalendar (ics) file and a vCard (vcf) file. Otherwise leave it blank.
This is an optional field. Enter Y (or y) if you want a vCard file and a vCalendar file written to the output folder specified above.

8. Then, Click on Start button to start the macro.

Please remember, you must set your Excel Security before running IPD Parse.
Special Considerations:
1. The IPD Parse macro is delivered with one Worksheet in the Workbook. That Worksheet must remain named: Main. (Other Worksheets will be added during processing).

Output Description: A. Output sub-system:


1) Worksheets added to the IPD_Parse Workbook. The following Worksheets will be generated if applicable: PIN Phone Call Logs Tasks Contacts SMS Calendar Memos DST Debug1 Debug2

2) A separate Workbook is created similar to the above but with only the DB report Worksheets included. The Main worksheet is omitted. The DST worksheet is omitted. All VB Components, i.e VBA code and user forms are omitted. This Workbook will be written to the specified folder. It will have a unique file name and may be retained for future use. 3) A composite text file containing reports for numerous DBs. The file name for the output composite text file is: IPD DB Reports.txt. The following reports will be generated if applicable: PIN Messages Tasks SMS Messages Memos Phone Call Logs Calendar Contacts CSV 4) Optionally, individual custom text files for vCard and vCalendar. The file name for the vCard file is: IPD vCard.vcf The file name for the vCalendar file is: IPD vCalendar.ics 5) Synopsis of Reports by Name by Format: Report Name __________ Contacts Report Format _________ Text

CSV vCard SMS Memos Phone Call Logs vCalendar Calendar Tasks PIN

XLS (>) Text vcf file Text XLS (>) Text XLS (v) Text XLS (v) ics file Text XLS (v) Text XLS (v) Text XLS (v)

B. Blackberry DBs
Various reports are generated based on the DBs detected on the input IPD file. The presence of the following DBs will generate reports: Address Book and Address Book Alll SMS Messages Calendar and Calendar All PIN Messages Memos Tasks Phone Call Logs

C. CSV format
A CSV formatted worksheet is produced containing all readable Blackberry fields from the Address Book data. This CSV data can be used as input to any subsequent application which uses CSV formatted address data. Standard Blackberry field heading names are used. This CSV data is also written to the composite text file. Comma separated value (CSV) files are used for exchanging data between various applications. Optionally, on the opening screen, you can select whether you want quotes around each output data element on the CSV file.

Some applications require data to be in quotes; while some do not allow quotes. WARNING !!! Any CSV formatted file must have quotes around every data element if there are any commas present in the data. Otherwise, the unintentional comma will be interpreted as a column separator and distort the the data/column alignment.

The CSV worksheet has been validated by successfully doing a Save As CSV format.file. If you have any questions, comments, or suggested changes to this output, please let me know. rapanui21@yahoo.com.
Reference: Comma-separated values; http://en.wikipedia.org/wiki/Comma-separated_values

D. vCard format
A vCard formatted file is optionally produced containing basic Address Book DB data. This vCard file can be used as input to any subsequent application which uses vCard formatted address data.

The vCard file has been validated as compliant with the vCard version 2.1 specification. If you have any questions, comments, or suggested changes to this output, please let me know. rapanui21@yahoo.com.
Note: Some applications require vCards to be written to individual files for use in downstream processing. Contact me if you have questions concerning this matter. rapanui21@yahoo.com Reference: http://en.wikipedia.org/wiki/VCard

E. vCalendar format A vCalendar formatted file is optionally produced containing basic Calendar DB data. This vCalendar file can be used as input to any subsequent application which uses vCalendar formatted data. The vCalendar file has been validated as compliant with the vCalendar version 1.0 specification.
Note: Some of the HHMMSS fields may have minor discrepancies. Start/End times are presented in local time not UTC based time.

Some applications require vCalendar to be written to individual files for use in downstream processing. Contact me if you have questions concerning this matter. rapanui21@yahoo.com

F. Miscellaneous
1. The IPD Parse Workbook is programmatically saved with changes at the end of the macro. This is done to ensure the output report worksheets are saved for your use. Another Workbook is created with only the DB reports Worksheets for your long-term use.

Trouble-shooting and Analysis:


Please follow the following steps if an exception condition occurs during the processing of IPD Parse:

Step 1
Please verify that the following are correct as described in the documentation: 1) Excel Security settings Run time error 1004 is a common symptom with incorrect Excel Security settings. 2) Input IPD file is created by Blackberry Desktop Manager. Error # 106 is a common symptom when a non-IPD file is input to the macro. 3) Input data specified on the opening screen. The IPD file must be the fully qualified filename of the IPD file (Path + Name). The Directory path specified must be an existing Directory. 4) Hardware and software used is as prescribed in the documentation. Specifically, Windows (XP or Win 7) and Excel (2003, 2007 or 2010)

Step 2
Contact the macro developer at rapanui21@yahoo.com .

Step 3
The macro developer may request you to rerun the macro with the Debug option activated. This option is described in the Debug Option on the sidebar of the online documentation. Please note, the Debug option introduces a lot of diagnostic overhead and the macro will run significantly longer than normal.

Acknowledgments:
Blackberry Developer Journals, January 2006, IPD File Format

Contributors:
Platinum Level: Jonathan Whitman, New Jersey SMS status. Hebrew and Russian language support. User interface design. Assistance with testing, design, consultation, and much, much more. Gold Level: Dinh Nho Hung, Vietnam Reports on Worksheets. Leading zero problem in phone number field. Silver Level: Steve Jackson, Virginia Addition of Address Book DB fields: Mobile2 and Home Fax. Gianluca Sonnino, Italy CSV with all readable Blackberry fields with Blackberry headings. Bronze Level: Doc Collins, Missouri Reported unique occurrence on IPD which had not been anticipated by the macro author.

Maximum Capacities and Limitations


As with most computer programs, there are some limitations in this macro. Most of the limitations are Excel limitations. Each of the following data elements have a limitation of 65,536: Contacts, Memos data lines (rows), Phone Call Logs data lines (rows), Calendar data lines (rows), PIN data lines (rows), Tasks data lines (rows).

An individual Excel cell can hold 32,767 characters. Excel has 65.536 rows and 256 columns. In my personal case, the data on my IPD file did not even come close to approaching any of these limitations. If you have or think you experience a limitation problem, contact me and we can discuss pursuing other alternatives.

FAQ: Q: Are there any special settings in Excel that need to be made to run IPD Parse ? A: Yes, and they are important. See the Excel Security settings section in this document for a description of these required settings.
Failure to do this will result in a VBA Run Time error 1004.

Q: A: Q:
A:

Do you charge a fee for the use of the software or for your services ? No. All my services are free. Will IPD Parse run on a Mac ?
I do not know. I have not tried it and I do not know anyone who has. If you try to run it on a Mac, please contact me with your results. I believe the IPD file produced by Blackberry Desktop Manager on a Mac is packaged in a .bbb file (similar to a Zip file).

Q:
A:

How long does IPD Parse run ?


It depends on the size of the input IPD file. My IPD is approximately 3 Mb and IPD Parse runs for about 40 seconds.

Q:
A:

Are reports generated for all Blackberry DBs ?


No, not all DBs. There are numerous DBs on the phone and consequently on the IPD file. Some DBs are inconsequential and meaningless to IPD Parse, such as Browser Cache DB. IPD Parse reports on most all user input data.

Q:
A:

What do I do if I have questions or comments about IPD Parse ?


Contact me at rapanui21@yahoo.com .

Q:
A:

What are the hardware/software requirements to run IPD Parse ?


See the Processing Platform section located elsewhere in this documentation.

Disclaimers:

This macro is for your personal use only. The author assumes no liability for your use of this macro. There is no AdWare, MalWare or hidden code present in this macro as distributed by the author. The author maintains Intellectual Property Rights of this macro. I declare that I am the original author of this macro. All of the code was developed by me. I further declare that the executable macro contained herein is in the public domain.

Error Conditions: Critical


Error 101: Worksheet integrity error. Invalid worksheet name. Error 105: File Open error. Error 106: DB Data Block is larger than 256 ^ 4 - 1 (255 255 255 255)

Description of Date / Time stamp fields: Phone Call Logs DB


Field Type: Length: Format: Metric: Adjustment factor: 4 4 8 byte; offset 3-10 (zero relative) Small endian Milliseconds since 1/1/1970 +/- GMT bias and DST

Example: 008 000 004 036 082 074 014 047 001 000 000 Where: 008 000 is: 4 is: 036 * 256^0 = 082 * 256^1 = 074 * 256^2 = 014 * 256^3 = 047 * 256^4 = 001 * 256^5 = 000 * 256^6 = 000 * 256^7 = =
1,301,614,842,404 total milliseconds from 1/1/1970 to date coded in IPD record.

(8 bytes) Field Type

Divide total milliseconds to determine days and fractional days. Add 25,569 (days from 1/1/1900 to 1/1/1970) Add/Subtract GMT bias; GMT +5 hrs would cause an addition of 5/24 or .2083333 days. Total Days since 1/1/1900 = 15,064.99 Format(Total Days Since 1/1/1900, "mm dd yyyy hh:mm") Result:

SMS Messages DB
Field Type: Length: Format: Metric: Adjustment factor: 1 byte; offset 16-23 (zero relative) Small endian Milliseconds since 1/1/1970 +/- GMT bias and DST 6 (DTSTART) and 7 (DTEND) 4 byte; offset 3-6 (zero relative) Small endian Minutes since 1/1/1900 +/- GMT bias, DST and fudge factor (+2 days)

Calendar DB
Field Type: Length: Format Metric: Adjustment factor:

Note: Date / Time stamp in Calendar DB records do not begin at 12:00am on 1/1/1900.

Tasks DB Field Type Field Offset (0 relative) Field Length Byte Format: Metric: Example:
Adjustment factor:

5&6 0-3 4 Small Endian Minutes since 1/1/1900


+/- GMT bias, DST and fudge factor.

PIN Messages DB Field Type Field Offset (0 relative) Field Length

TBD TBD TBD

Byte Format: Metric: Example:


Adjustment factor:

Small Endian TBD


+/- GMT bias and DST

Anyone who wants to discuss date/time stamp stuff, please contact me at rapanui21@yahoo.com

Known discrepancies in IPD Parse macro:


Every attempt has been made to provide an error free macro. If you encounter any problems, please report them to me. At present, the following known discrepancies exist: Ver 2.1 1. vCalendar fields hhmmss sometimes have a small discrepancy. 2. Date/time stamp not in PIN Messages report. The date/time stamp field has not been located on the IPD.

You might also like