Professional Documents
Culture Documents
IPD Parse
Mike West
Ver. 2.2.8
rapanui21@yahoo.com
June 2012
Macro Description: The IPD Parse macro is designed to read a Blackberry IPD file and create reports for certain data bases detected on
the IPD file. These reports are written to Excel Worksheets (within the IPD Parse Workbook), a separate Workbook with only the DB report worksheets, a single composite text file, and optionally to some individual text files. IPD Parse runs on Windows and Excel. The input IPD file must be created by the Blackberry Desktop Manager program. The IPD file may be a comprehensive file containing a full backup with all Blackberry DBs present or it may be a customized file created with the Advanced option of the Backup function and contain a single DB. Without a macro of this nature, you would have limited options to view or migrate your Blackberry information from your phone to a non-Blackberry platform. This macro helps you view and use various portions of the otherwise unreadable binary IPD file.
Background:
While I do not provide any contractual technical support, I would be happy to assist you in any manner. I do not charge for my services. All my services are free. Please feel free to email me (rapanui21@yahoo.com). This macro represents hours and hours of research, coding, debugging, testing, and writing documentation. I did this as a labor of love. My aim is to provide you with a valuable tool.
Development Platform:
Following is the hardware/software I used to develop this macro. Windows XP, Excel 2003, Visual Basic 6.3,
Blackberry Curve 8310, Blackberry Desktop Manager V5.0. Binary Viewer; Freeware Version 2.0.9.1214
Processing Platform:
Following is the hardware/software on which this macro has been successfully tested. Windows XP, 2000, Win 7 NOTE: This macro does not run well on Vista. Excel 2003, 2007, 2010 Visual Basic 6.3, Blackberry Curve, Pearl, Torch, Bold Blackberry Desktop Manager V5.x, V6.x
Revision History:
Ver 1.7 (June 2011) Ver 1.8 (August 2011) Ver 2.2 (February 2012) Ver 2.2.4 (March 2012) Ver 2.2.7 (May 2012) Ver 2.2.6 (June 2012)
Getting Started:
There are three important pre-requisites to running the IPD Parse macro.
1.
Input Preparation
A. B. IPD file DST file
2. 3.
If you are using BlackBerry Desktop Manager V4.1 to 5.0.1, see Article ID: KB12487 in the BlackBerry
Knowledge Base.
If you are using BlackBerry Desktop Manager V6.0, see Article ID: KB23680 in the BlackBerry Knowledge Base. (Google 'BlackBerry KB23680). Note: If performing a Selective Backup, enter'No' when asked if you want 'Encryption'.
The IPD Parse macro provides calculations for Daylight Saving Time (DST), also referred to as Summer Time, for all time zones world-wide (as appropriate).
B. DST file
USA
DST calculations for the USA are based on the appropriate United States Federal law. These calculations are performed automatically and internally by the macro with no actions required by the user. From 1986-2006, DST begins at 2 a.m. on the first Sunday of April and ends at 2 a.m. on the last Sunday of October. From 2007-current, DST begins at 2 a.m. on the second Sunday of March and ends at 2 a.m. on the first Sunday of November. The USA States of Hawaii, Indiana, and Arizona have and/or had DST rules different from the USA Law. The optional override should be utilized, as appropriate, for these locations.
Non-USA
Since there are so many variations to DST rules around the world, IPD Parse provides for an optional override where users can declare the DST dates to be used for their location. Any user whose non-USA location does not observe DST may run the macro without the presence of the optional DST override file. This will cause all DST calculations to be bypassed. Any user whose non-USA location does observe DST should create and make available the optional DST override file. This will cause all DST calculations to be made using those specified rules.
IPD DST.xls
File Location:
IPD DST.xls must reside in the same directory path as the input IPD file.
Worksheet:
DST data must be in the first worksheet of IPD DST.xls.
File Format:
Column A is the date/time of the Start of the DST period. Column B is the date/time of the End of the DST peroid. The date/time may be entered in any valid Excel date/time format. Successive entries are entered on successive rows. No blanks rows should be entered. No column headings should be entered. Enter a separate row for every year represented on your input data.
Example:
Row #1 Row #2 Row #3 Row #4 Row #5 Row #6 Row #7 Row #8 Col A ---------------------4/30/2006 02:00 AM 3/11/2007 02:00 AM 3/9/2008 02:00 AM 3/8/2009 02:00 AM 3/14/2010 02:00 AM 3/13/2011 02:00 AM 3/11/2012 02:00 AM 3/10/2013 02:00 AM Col B -----------------------10/29/2006 02:00 AM 11/4/2007 02:00 AM 11/2/2008 02:00 AM 11/1/2009 02:00 AM 11/7/2010 02:00 AM 11/6/2011 02:00 AM 11/4/2012 02:00 AM 11/3/2013 02:00 AM
Note: The dates in the above example are actual USA DST start and end dates.
----------------------------------------------------------------------------
2.
Excel, by default, does not allow macros to run. You must change your Excel Security settings to allow IPD Parse (or any macro) to run. Failure to do this will result in a VBA Run Time error 1004.
Excel 2003
1. Select Tools -> Macro -> Security Select the Security Level tab. Select either Medium or Low and Press OK. 2. Select Tools -> Macro -> Security Select the Trusted Publishers tab. Place a checkmark to select Trust access to Visual Basic Project.
Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.
Excel 2007
1. You must save the IPD Parse with an xlsm extension to indicate it is a macro-enabled workbook. Save As, change the type to Excel Macro-Enabled Workbook. 2. Select the main Excel menu (by selecting the Excel logo on the top left of the spreadsheet). At the bottom right of this menu, select Excel Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. From within the Macro Settings option, select Enable all macros How about the 2nd & 3rd option? Also, place a checkmark to select Trust access to the VBA project object model. Then Press OK. Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.
Excel 2010
1. You must save the IPD Parse with an xlsm extension to indicate it is a macro-enabled workbook. Save As, change the type to Excel Macro-Enabled Workbook. 2. Select the File tab, then select Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. From within the Macro Settings option, select Enable all macros . How about the 2nd & 3rd option? Also, place a checkmark to select Trust access to the VBA project object model. Then Press OK. Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.
Excel 2007
Select the main Excel menu (by selecting the Excel logo on the top left of the spreadsheet). At the bottom right of this menu, select Excel Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. Select the Trusted Locations option from the left and menu. Press Add new location and follow the prompts.
Excel 2010
Select the File tab, then select Options. From the window that pops up, select Trust Center option. Click on Trust Center settings. Select the Trusted Locations option from the left and menu. Press Add new location and follow the prompts. Note: After you change Excel Security settings, you must re-start Excel for the new settings to take effect.
3.
4. Enter Y to have quotes placed around fields on CSV data. Otherwise leave it blank.
This is an optional entry. Some applications require quotes around data fields. Enter Y (or y) if applicable. See CSV Tutorial for details.
5. Click on drop down menu if you want to select a single report to be written to text file. Leave blank to write All Reports.
This is an optional entry. If you want only one report written to the composite text file, select the name of that report. Valid Report names are: Memos, Calendar, Contacts, Phone Call Logs, SMS, PIN, Tasks, and CSV) This option may be used, for example, when you want just the CSV data for future downstream processing.
6. Enter Y to write SMS report to the text file only and not to the xls SMS worksheet. Otherwise leave it blank.
This is an optional entry. This option may be used, for example, when you have (or think you may have) greater than 65,536 SMS messages on your IPD file (which is an Excel row limitation). This would allow you to view your entire SMS report on the text file. Enter Y (or y) if applicable.
7. Enter Y to create a vCalendar (ics) file and a vCard (vcf) file. Otherwise leave it blank.
This is an optional field. Enter Y (or y) if you want a vCard file and a vCalendar file written to the output folder specified above.
Please remember, you must set your Excel Security before running IPD Parse.
Special Considerations:
1. The IPD Parse macro is delivered with one Worksheet in the Workbook. That Worksheet must remain named: Main. (Other Worksheets will be added during processing).
2) A separate Workbook is created similar to the above but with only the DB report Worksheets included. The Main worksheet is omitted. The DST worksheet is omitted. All VB Components, i.e VBA code and user forms are omitted. This Workbook will be written to the specified folder. It will have a unique file name and may be retained for future use. 3) A composite text file containing reports for numerous DBs. The file name for the output composite text file is: IPD DB Reports.txt. The following reports will be generated if applicable: PIN Messages Tasks SMS Messages Memos Phone Call Logs Calendar Contacts CSV 4) Optionally, individual custom text files for vCard and vCalendar. The file name for the vCard file is: IPD vCard.vcf The file name for the vCalendar file is: IPD vCalendar.ics 5) Synopsis of Reports by Name by Format: Report Name __________ Contacts Report Format _________ Text
CSV vCard SMS Memos Phone Call Logs vCalendar Calendar Tasks PIN
XLS (>) Text vcf file Text XLS (>) Text XLS (v) Text XLS (v) ics file Text XLS (v) Text XLS (v) Text XLS (v)
B. Blackberry DBs
Various reports are generated based on the DBs detected on the input IPD file. The presence of the following DBs will generate reports: Address Book and Address Book Alll SMS Messages Calendar and Calendar All PIN Messages Memos Tasks Phone Call Logs
C. CSV format
A CSV formatted worksheet is produced containing all readable Blackberry fields from the Address Book data. This CSV data can be used as input to any subsequent application which uses CSV formatted address data. Standard Blackberry field heading names are used. This CSV data is also written to the composite text file. Comma separated value (CSV) files are used for exchanging data between various applications. Optionally, on the opening screen, you can select whether you want quotes around each output data element on the CSV file.
Some applications require data to be in quotes; while some do not allow quotes. WARNING !!! Any CSV formatted file must have quotes around every data element if there are any commas present in the data. Otherwise, the unintentional comma will be interpreted as a column separator and distort the the data/column alignment.
The CSV worksheet has been validated by successfully doing a Save As CSV format.file. If you have any questions, comments, or suggested changes to this output, please let me know. rapanui21@yahoo.com.
Reference: Comma-separated values; http://en.wikipedia.org/wiki/Comma-separated_values
D. vCard format
A vCard formatted file is optionally produced containing basic Address Book DB data. This vCard file can be used as input to any subsequent application which uses vCard formatted address data.
The vCard file has been validated as compliant with the vCard version 2.1 specification. If you have any questions, comments, or suggested changes to this output, please let me know. rapanui21@yahoo.com.
Note: Some applications require vCards to be written to individual files for use in downstream processing. Contact me if you have questions concerning this matter. rapanui21@yahoo.com Reference: http://en.wikipedia.org/wiki/VCard
E. vCalendar format A vCalendar formatted file is optionally produced containing basic Calendar DB data. This vCalendar file can be used as input to any subsequent application which uses vCalendar formatted data. The vCalendar file has been validated as compliant with the vCalendar version 1.0 specification.
Note: Some of the HHMMSS fields may have minor discrepancies. Start/End times are presented in local time not UTC based time.
Some applications require vCalendar to be written to individual files for use in downstream processing. Contact me if you have questions concerning this matter. rapanui21@yahoo.com
F. Miscellaneous
1. The IPD Parse Workbook is programmatically saved with changes at the end of the macro. This is done to ensure the output report worksheets are saved for your use. Another Workbook is created with only the DB reports Worksheets for your long-term use.
Step 1
Please verify that the following are correct as described in the documentation: 1) Excel Security settings Run time error 1004 is a common symptom with incorrect Excel Security settings. 2) Input IPD file is created by Blackberry Desktop Manager. Error # 106 is a common symptom when a non-IPD file is input to the macro. 3) Input data specified on the opening screen. The IPD file must be the fully qualified filename of the IPD file (Path + Name). The Directory path specified must be an existing Directory. 4) Hardware and software used is as prescribed in the documentation. Specifically, Windows (XP or Win 7) and Excel (2003, 2007 or 2010)
Step 2
Contact the macro developer at rapanui21@yahoo.com .
Step 3
The macro developer may request you to rerun the macro with the Debug option activated. This option is described in the Debug Option on the sidebar of the online documentation. Please note, the Debug option introduces a lot of diagnostic overhead and the macro will run significantly longer than normal.
Acknowledgments:
Blackberry Developer Journals, January 2006, IPD File Format
Contributors:
Platinum Level: Jonathan Whitman, New Jersey SMS status. Hebrew and Russian language support. User interface design. Assistance with testing, design, consultation, and much, much more. Gold Level: Dinh Nho Hung, Vietnam Reports on Worksheets. Leading zero problem in phone number field. Silver Level: Steve Jackson, Virginia Addition of Address Book DB fields: Mobile2 and Home Fax. Gianluca Sonnino, Italy CSV with all readable Blackberry fields with Blackberry headings. Bronze Level: Doc Collins, Missouri Reported unique occurrence on IPD which had not been anticipated by the macro author.
An individual Excel cell can hold 32,767 characters. Excel has 65.536 rows and 256 columns. In my personal case, the data on my IPD file did not even come close to approaching any of these limitations. If you have or think you experience a limitation problem, contact me and we can discuss pursuing other alternatives.
FAQ: Q: Are there any special settings in Excel that need to be made to run IPD Parse ? A: Yes, and they are important. See the Excel Security settings section in this document for a description of these required settings.
Failure to do this will result in a VBA Run Time error 1004.
Q: A: Q:
A:
Do you charge a fee for the use of the software or for your services ? No. All my services are free. Will IPD Parse run on a Mac ?
I do not know. I have not tried it and I do not know anyone who has. If you try to run it on a Mac, please contact me with your results. I believe the IPD file produced by Blackberry Desktop Manager on a Mac is packaged in a .bbb file (similar to a Zip file).
Q:
A:
Q:
A:
Q:
A:
Q:
A:
Disclaimers:
This macro is for your personal use only. The author assumes no liability for your use of this macro. There is no AdWare, MalWare or hidden code present in this macro as distributed by the author. The author maintains Intellectual Property Rights of this macro. I declare that I am the original author of this macro. All of the code was developed by me. I further declare that the executable macro contained herein is in the public domain.
Example: 008 000 004 036 082 074 014 047 001 000 000 Where: 008 000 is: 4 is: 036 * 256^0 = 082 * 256^1 = 074 * 256^2 = 014 * 256^3 = 047 * 256^4 = 001 * 256^5 = 000 * 256^6 = 000 * 256^7 = =
1,301,614,842,404 total milliseconds from 1/1/1970 to date coded in IPD record.
Divide total milliseconds to determine days and fractional days. Add 25,569 (days from 1/1/1900 to 1/1/1970) Add/Subtract GMT bias; GMT +5 hrs would cause an addition of 5/24 or .2083333 days. Total Days since 1/1/1900 = 15,064.99 Format(Total Days Since 1/1/1900, "mm dd yyyy hh:mm") Result:
SMS Messages DB
Field Type: Length: Format: Metric: Adjustment factor: 1 byte; offset 16-23 (zero relative) Small endian Milliseconds since 1/1/1970 +/- GMT bias and DST 6 (DTSTART) and 7 (DTEND) 4 byte; offset 3-6 (zero relative) Small endian Minutes since 1/1/1900 +/- GMT bias, DST and fudge factor (+2 days)
Calendar DB
Field Type: Length: Format Metric: Adjustment factor:
Note: Date / Time stamp in Calendar DB records do not begin at 12:00am on 1/1/1900.
Tasks DB Field Type Field Offset (0 relative) Field Length Byte Format: Metric: Example:
Adjustment factor:
Anyone who wants to discuss date/time stamp stuff, please contact me at rapanui21@yahoo.com