Vitor Rodrigues

This is my problem: I have a huge xml file (150MB), in which I want to rename some of the node names. Conisder the example:

<root><prefix_name1>1</prefix_name1><prefix_name2>text</prefix_name2></root>

The creators of this document were not aware of namespaces, so they decided to use different prefixes in the element names in the documents created. My goal is to rename all the tags with prefix removing the "prefix_" part of the name. The solution looked simple, but it isn’t.

To begin with, I tried a sed, using the argument "s/prefix_//g". This didn’t work, because in AIX sed only accepts up to 4096 bytes per line (I read somewhere that this doesn’t happen with some sed versions, but I couldn’t spend the whole day trying versions of sed). Using perl also resulted in Out of memmory errors (I’ve just very basic perl knowledge, so there may be a way of handling such big lines).

So, I had to come up with a set of steps to do this change and maintain the file in it’s original format (all contents in one line). The steps where:

•  used the tr utility to translate all ">" characters into a new line (a new > is printed before the replaced char) using tr  ‘>’ ‘\012′ < $1  | while read line; do echo "${line}>"  >> $1.new done.
• rename the tags by removing the prefix. Either use sed or perl with the replacement pattern "s/prefix_//g".
• delete all the "\n" (newline chars) from the current file, to get back to the one liner format. Used perl to read from one file and write to other applying "s/\n//g" to the content.

I’m sure that this is not the best solution, and there must be some utility out there that I can use so, if you have any idea, just leave a comment.

Popularity: 3% [?]

IBM Lotus® EasySync® Pro lets you synchronize your IBM Lotus Notes® e-mail, calendar, tasks, contacts and journal information between Lotus Notes and supported market-leading personal data assistants (PDAs).

I was looking for this tool for months and only today I found it by accident. Dumb me! I should have guessed that we had something like this.

It works great! Although Lotus Notes and Windows Mobile calendars use different structures for events, there was no loss of information at all and the synchronization worked both ways. For now, I synchronized only my calendar events. I don’t use my PDA to do email yet. Besides the phone and GPS capabilities, what I use most is the calendar, taks and notes. 

Popularity: 3% [?]

Originally uploaded by Vitor Rodrigues.

For the first time in many months, I really gave some rest to my laptop during my vacations. Last week was spent mostly with family, girlfriend and friends. So, only now I got to do the review of the year 2006:

•  started the year looking for a regular job to start after the end of my internship at IBM in Silicon Valley (my internship was going until August, but Visa applications need to be filed by April, so you need to have everything set by then).
• after a few job interviews, got job offers from both IBM and eBay (received both within a 20 minute interval ).
• decided to accept IBM’s offer. Although eBay’s offer was economically better, I really enjoy the work environment at IBM, I have most of my friends there, and more important than that, I was proposed new job responsabilities that made me prefer this position.
• presented at some internal conferences about using a Wiki environment in a big scale software testing project.
• presented at NPUC about using Wikis to give power to team members in the creation of an information repository for cross-team utilization.
• hired as a regular employee in March, 5 months before the end of my internship.
• published a IBM white paper on XML Indexing in DB2 9 pureXML.
• decided to accept an opportunity for a 1-year international assignment at IBM Toronto Laboratory in Canada, from August 2006 to August 2007.
• after 18 months in California, I finally enjoyed some vacations in Portugal, visiting some places I hade never been before, like Tomar, Sintra, Evora and Vila Vicosa.
• move to Canada.
• co-lectured a workshop at CASCON 2006.
• visited New York for the first time.
• published a techincal paper in the IBM developerworks website.
• finished another soon to be published white paper also related with DB2 pureXML.
• contributed to the DB2 pureXML demo.
• came home for Christmas and New Year.

Lots of stuff happening, mostly at work. I have the feelling that I could have done more, but I’m still happy with the results. I did more/better than what I was expecting one year ago.

In the personal life, besides the long awaited vacations in Portugal, all the other events where caused by work or work related (look for job, move to canada, trip to NY).

 For 2007:

• professionally: improve presentation, communication, organization skills.  Find time to invest in side/innovative projects. Read more.
• personally: more local exploring. Sometimes there are lots of interesting stuff in the place we live that we ignore and only later we realize how interesting that place was. Start to read. Save more money (hint: no more gadgets ).

Popularity: 11% [?]

Flickrfox is a nice Firefox extension that lets you browse Flickr photos and displays them in a sidebar. You can choose to display photos from your stream, your groups, everyone else photos and from the explore/interestingness page. I currently have it displaying the most interesting photos. It’s a nice way of filling up the microbreaks during the day.

Popularity: 2% [?]

Originally uploaded by SkyShaper.

Popularity: 5% [?]

While I was trying to create a stored procedure in my DB2 Express-C windows installation, I got the following error message:

DB21034E The command was processed as an SQL statement because it was not a valid Command Line Processor command.  During SQL processing it returned:SQL0219N The required Explain table "SCHEMA.EXPLAIN_INSTANCE" does not exist. LINE NUMBER=16.  SQLSTATE=42704

At first, this error message didn’t make much sense. Why would DB2 need to use the explain tables if I’m not running this command in the explain mode? Even doing a db2 set current explain mode no didn’t seem to help. But, creating the explain tables would solve the problem (you can create the explain tables by hand by runing db2 -tvf ~/SQLLIB/MISC/EXPLAIN.DDL). If creating the explain tables would solve the problem, it was because somehow, DB2 was running in the explain mode when creating the stored procedures.

Investigating the DB2 registry variables (db2set -all) I noticed the existence of the variable:

 [i] DB2_SQLROUTINE_PREPOPTS=explain yes

A search for this term on DB2 documentation and the answer was found. The variable DB2_SQLROUTINE_PREPOPTS is used to specify the precompile and binding options for store procedures. After knowing that was the problem, the fix is easy. Just update the registry variable running db2set DB2_SQLROUTINE_PREPOPTS="explain no".

Note: Since this is an instance level variable, the changes will only make effect after you restart your DB2 instance. Just run db2stop and db2start and you’re good to go!
 

Popularity: 7% [?]

More Pics:New York City Flickr Set.

Update: Apple Store and Empire State Building. 

Popularity: 7% [?]

Although Joomla is one of the most advance CMS available, there are some things that I really don’t like/agree with. One of these things is the complex Access Control List. There are too many different user types and registered users have no permission to add content to the site. They need to be upgraded from “Registered” to “Author”. Still, and Author can only create content, but not publish it. Only a Publisher can publish content. And between Author and Publisher there is still the Editor, that can create and edit content, but can’t publish. As you can see, there are too many different user types (and I forgot to mention the Manager and Administrator types also availabel for the frontend. On the backend, another long list exhists).

If this complex organization was not enough, you cannot specify a default user type, so everybody that registers becomes a “Registered User”, without permissions to add content. The Joomla solution is to go to the administration backend  and switch the user to the group you think it fits (one of the listed above). However, if you want a user to be able to create, publish and edit it’s OWN content, it’s not possible rigth now. Registered users can only create content and Publishers can publish content from all users, so none of this types is good for our needs. After reading on foruns about the Joomla ACL (that was inherited just like it is now from Mambo) and about the no-will of Joomla developers to listen to community and change it, the only solution is to change the source code yourself. A very easy solution indeed. Just open the file includes/gacl.class.php and inside the funciton gacl add the lines:

$this->_mos_add_acl( 'action', 'add', 'users', 'registered', 'content', 'all' );
$this->_mos_add_acl( 'action', 'edit', 'users', 'registered', 'content', 'own' );
$this->_mos_add_acl( 'action', 'publish', 'users', 'registered', 'content', 'own' );

This will allow registered users to add any type of content, edit their own content and publish their own content. The same logic applies to other user types in case you want to change any of the permissions on the file.

Auto Publishing

Another sometimes useful feature not available in Joomla is the frontpage auto-publish of content (the Publisher is the one that decides to put the content in the frontpage or not when he publishes it). This can be achieved with a single change in the file components/com_content/content.php. Search for the line:

$row->frontpage         = 0;

and replace it with:

$row->frontpage         = 1;

This will enable frontpage auto-publishing of all your contents. Note: although this may be useful for publishing news in the frontpage, it may not be what you want for when you create articles that are not to display in the frontpage (In this case, you need to uncheck the frontpage publishing box). You should consider wich one is the most common case and decide on the best option for it.

Popularity: 100% [?]

After my yesterdays participation in the workshop DB2 XML – A True XML and Relational Database, the rest of the week will be spent at the CASCON Technology Showcase, with a poster and a some live demos aout DB2 9 pureXML™.

Yesterday’s workshop as a success! The room was completely full and we even had some people having lunch in the meeting room just to have more time to explore the project and ask us questions. My session was the first in the afternoon, entitled "XML Indexing in DB2", with a 30 minute presentation plus 40 minutes of lab exercises. As my first experience lecturing at a workshop, it was interesting to be on the other side and feel the responsability (and excitment) of having about 40 people staring at us and asking questions. Most of the public in our workshop were wither DB Appllication developers or students in the database field.

Note 1: for as much that you prepare a presentation, there will be always some question that you don’t expect, and that if have no idea what’s the correct answer.

Note 2: for as much that you prepare the lab exercises, there will always be a typo on it Nothing serious.. just a "$d" not escaped being consumed by the linux command line.

Note 3: the food at Hilton is great!

Popularity: 4% [?]

12 hours of flight can be transformed into 32 hours of travelling, if you have to "visit" 4 airports and one of the flights gets delayed, making you spend the night at the airport waiting for a connection flight. The (not!) awesome United Airlines declined to pay for a hotel room for the about 50 people with lost connections, because, according to them, the delay "was in the taxi way, so it’s not our fault". Well, for some reason that wasn’t explained, the airplane left the gate in direction to the taxi way almost one hour late. Considering that my connection was 15 minutes after our arrival to Washingotn Dulles Airport, that hour would have been enough for me!

After leaving Carapeços (Barcelos, Portugal) at 8:00h of September 25, I just arrived Toronto (Canada) at 11h30 of September 26.

I’ll be in the Toronto area for one year, enjoying the freezing weather of Canada, and also enjoying my work at IBM Canada, working as a technology enabler for DB2 9 pureXML™.

1st impressions:

• traffic is hell! Even worse than Los Angeles.
• The streets/houses are old and not taken care off.
• the IBM lab is just beautifull, and with excellent conditions.
• I need to get a network cable, because the wireless signal in my office is very weak…

Popularity: 11% [?]