0.3 Release… Multi-touch with an Extension on/off switch

Some great achievements here…  and some utter frustration.  After spending an hour due to missing some curly braces on a call to a property that plays host to an anonymous function, I finally got a very cool extension in place to test the waters of intercepting and tinkering with the default behaviour of the multi-touch events that I have in place.  To get these noted first… here are some pertinent links:

The almost complete Release…

I despise this.  But I must accept.  We discussed in class some of the concepts around open source development and one of the ideas was the relaxed pace to get things done… the lack of stress and pressures that come from a normal job.  Who wants to be normal anyhow…. well…. being in school with deadlines still makes it feel pretty normal, but here it goes.

The Patch

I managed to use some of the latest coding found in the firefox realm to fire off the events for the multi-touch actions.  But the new code has a strange behaviour that causes the x/y coordinates of the event to get lost for my up/down swipe events.  So I set up the action to use the older method of firing the event.  Actually, all of the functionality could work fine under the older system, but I wanted to have the newer code implementations in place for the sake of future proofing the work and taking it in a direction they are heading in anyhow.  So for that it feels a little incomplete, but it still is only a 0.3 release and not a full fledge polished submission.

The Extension

The extension was a bugger.  I’m not even sure I went in the correct direction with it, but I’ve got it doing something cool.  First here’s the problem with the extension.  It won’t work when you install it the normal way.  You have to expand the files and then use a reference file that you place in your extensions directory of your profile… and then it will work.   So I’m looking into that, but for the sake of posting it even though  its not perfect… here it is!

What’s cool about it is this.  In the menu that the “add-ons” are on, this extension puts two items on the bottom of the menu list.  One will disable or turn off the swipe and pinch behaviours and the other will turn them back on!!!  I thought it was fun.  It does so by storing the preferences when Thunderbird loads, and then erases them when you turn them off, and places them back in place when you turn the mutli-touch actions back on.  Just to protect the settings, they are saved back when Thunderbird quits.

My digression

Well, I’m not back next semester, but I will approach this bug as a true open source project… and not a school project.  Which means I’ll relax with it and work on it when I can.  I do like what I’m working on and at the moment have a wish to do OSD700 in the summer if possible, so keeping my feet wet will be good.  I will continue to post blogs, but not likely as frequently as we’ve been doing or trying to do.  Thanks for wonderful semester and I’ll see you all around.

Posted in Open Source, _Old Archived Posts | Leave a comment

Multi-touch support… next step done

In terms of my open source course, my latest patch to the Mozilla Thunderbird community is a little behind.  In terms of the Thunderbird community, I’ve already received an email (less than 5 minutes from posting the patch) that they’re excited about what I’m doing.  Such gratitude is certainly humbling.  Its nice to spend a couple straight days 8-10 hours straight each time and pull out a patch that someone is just really happy that somebody is doing the work.  Kind of a nice warm fuzzy feeling…

The patch is here.

This is not my final release blog for my 0.3.  It is a first major stage of it.  This blog is simply to commemorate the occasion.  Will post again tomorrow.  Until then, I’m working with buddhatron: on the extension’ing of this patch.

Posted in Open Source, _Old Archived Posts | Leave a comment

Mozilla Thunderbird Project follow up

Onwards to 0.3

Ok, so the semester is nearing an end and the multitude of intense courses this semester has been hitting me to the ground…. but I shake off the dirt from my clothes, get my composure… and keep pushing forward.  So where are things at and where are they going.

Blogs for my project
0.2 Release
Detecting Window Panes
0.1 Release

I have a problem

The first realization… I’m a perfectionist… well, its possible and likely… but more importantly, I’ve found myself and received feedback from people that I program by playing it safe by doing a lot of planning.  Whereas those that gave me the feedback attributed their own programming style to be more like a bull in a china shop.  How does this relate?

I tend to plan my approach a lot and I like to know what I’m doing.  The plan isn’t such a bad idea and I’ll note it next.  The whole “knowing what I’m doing” thing, just isn’t going to happen while I’m in school…. hell, that won’t even happen while I’m on the job.  It’ll happen after a project is done… and then I’ll move onto the next project and the learning starts all over.

What improves, and what I’m improving at, is my general understanding of programming and development work.  So I suppose the problem is not so much not knowing what I’m doing… but ensuring that I’m trying things without being so  concerned whether its going to work perfectly the first time.  Now that I’m on the crunch of the semester end… this type of approach IS A MUST!

My Plan

  • Fix the scrolling of the message pane so that it does a full scroll top/bottom and not paging.
  • create an extension to override the default behaviour (of course this will require sweetening up the current code that’s in place)
    • getting preferences configured so that the flow of the code is streamlined enough it can be capture for multiple events from a single point.
    • get familiar with extensions in Thunderbird, particularly for capturing user multi-touch events. (Nadim is aiding with this via blogging right now)
    • build the extension and cleaned up code and submit a patch and post on progress.

There is much learning to be done here.  So for now, the trick will be to get the learning accomplished and get coding as I figure things out.  Even if it means a change in the code could break the whole thing.  Just dig in!

Posted in Open Source, _Old Archived Posts | Leave a comment

Mozilla XPCOM Learning Lab

Here is another fine presentation for learning XPCOM from Mozilla in my Open Source program at Seneca College.  This lab guides you through the process of creating an extension as part of the mozilla build.  You can also build extensions as a separate XPI installer which we did in a previous lab and I posted on by inflicting a tabbing behaviour that opened new tabs to the left of the current tab instead of the right hand side.

The Extension in Action

Above, is a picture of the extension that was added to the Firefox build.  Actually, the XPCom class that we built only has 1 attribute which a getter/setter are made and a method called add( ) which takes two number parameters and adds them together.  Also, when those methods/properties are called upon, they produce output into the debugging log which is what you see in the command line output at the bottom of the picture.  Seems simple yes!… Why did it take me a week to figure it out then.

Lost in File path Jungle

One of my reasons for taking so long with this is due to the fact that when the lab was originally written, the place where the header files for the automatically created XPCom object had moved since then.  They are now stored in a different place.  This was discovered when we asked the instructor in class and he pointed this out.  It didn’t mean that we still weren’t stumped on trying to navigate our way around.

Contributions to the Learning Wiki

In order to rectify where the heck the files were being created automatically in the lab, I had to straighten out the wiki page that hosted this lab assignment.  My teacher invited it to be updated when we found the error, so just recently I went in and did that.

Essentially, in the lab, it was stating that the header files would be placed in a directory that was no longer being created within “$(moz_obj)/dist/include/{noLongerUsedDirectory}”.  So I went through the article and made changes as they appear in the image above.

All of the files that were dynamically created were in the mozilla source code directory and then inside the object directory where all of your compiled object files are stored.  The difference now is the in the “dist/include/” directory it now has an alias to the header file.  The actual header file is placed in “extensions/{extensionName}/public/_xpidlgen/”.  So I went through the lab and made changes to the file paths to reflect those changes.  Hopefully this will help clear things up for others in subsequent courses.

Oh ya, Install the Extension!

Ok, so I needed help from my instructor and got some ideas on why my code wasn’t working.  First thing that he had me check was to see if the extension was installed.  At that moment I recognized a void in my thinking…. I realized I had not even thought that this extension would need to be installed at all.  I figured… its built in the source code and compiled with Firefox.  I thought it’d be installed automatically.  Not the case!

So for the sake of sharing this simple little tidbit with others, maybe it’ll help make more sense of extensions that are built in the browser.  Also note… the Extension Developer extension will give you that javascript shell that the lab will need too.

Posted in Open Source, _Old Archived Posts | Leave a comment

Thunderbird Multi-touch 0.2 Release

Well after much fussing about with porting a bug fix that was intended for Firefox in dealing with multi-touch scrolling issues into  Thunderbird, I resolved the process and now have 3 finger swipe scrolling.  In the end, the final fix was removing a shortcut of “Ci” that was used in Firefox code to represent “Components.interfaces”.

My patch is posted here, and a sample package of Thunderbird for Mac with my patch in place is here.

After an excellent introduction to components and their idl interfaces in class today, this fix was almost simple.  In turn, this got the function that a firefox developer came up with which was to scroll a window by synthesizing a mouse scroll event.

Previously I had found a way to isolate out the 3 different panes in the Thunderbird main window…. being:  Folder pane, Message thread pane, and the Message pane.  In the end this wasn’t necessary, but it may come in handy for other custom actions.

Currently, preferences for these events aren’t propagating through Thunderbird properly and I have to learn the preferences system in Thunderbird to get it in use.  I imagine this will help to slim out the code and ensure better integration for extension/add-on capabilities for customizing the multi-touch behaviors.

There also seems to be a change in the behaviour of the synthensized mouse event that I am calling.  Since updating my revisions, the forced scroll behaviour now only seems to do a pageUp/pageDown event instead of a full scroll Top/Bottom.  I will need to investigate and maybe explicitly synthesize pageTop/pageBottom key strokes.

Posted in Open Source, _Old Archived Posts | Leave a comment

Detecting a Pane in Thunderbird!

From failure, to opportunity, to failure, to a success that will hopefully lead to another success… but inevitably, the cycle will continue!

What is all this referring to?   My work on implementing multi-touch controls on Thunderbird. It’s a bug in the Mozilla community and its a major project for my Open Source Development course.  You can also follow this on my project wiki page.

The Starting Point of this Phase

My 0.1 release acts as the starting point that I am referencing here.  Back then, I managed to get a 3 finger swipe in all directions to make an action over the 3 pane message window in Thunderbird.  The 3 page message window consists of a pane on the left for showing the folders in your email account, a pane at the top that is the thread of messages for the selected folder, and a pain below that that shows the selected message… no, that’s not a spelling error!

So although there was an action for all 4 swipe direction, it was the same action that impacted the same pane, regardless of which pane your cursor was hovering over.  The community concluded, and I agreed, that a distinct action should occur that impacts the pane that your are over top of.  So… this is where it begins.

A Possible Solution from a Firefox Bug

Along came a spider who sat down beside her and said…”check out this bug, its scrolling the page without requiring a focus event!”.  Ok ok…. it’s a little late at night and I’m writing blogs again… but seriously, out of what seemed like now where, I received an email from someone adding a “polished” whiteboard comment to a bug that I happen to have been following.

I was considering that maybe for my solution in Thunderbird, I need to do something that wouldn’t hold a focus to the content window itself (which contains all of the panes) and instead NOT hold a focus to the window in lieu of retaining it for one of the panes alone.  When I saw the bug linked above, they were trying to remove focus on elements on a webpage and thus allow the parent element to scroll so that the scroll event doesn’t get trapped by the child.

Try this  (… if you can)

You can try this if you have a mac with a multi-touch trackpad… Open up Firefox (I know, you are already using it, so this is for those who don’t KNOW yet….😉 …. and:

  1. Go to the Google Homepage (www.google.ca).
  2. Narrow the height of the window so that your vertical scroll bar appears.
  3. Place your cursor inside the textbox where you’d type in your search for Google.
  4. Now try and scroll by sliding up or down with 3 fingers on your trackpad.
  5. Witness the fact that it does not scroll the page at all!
  6. Now remove your cursor from the textbox by touch the Firefox image.
  7. Now scroll again as you did above… it scrolls!
  8. The solution that they created fixes this so that it scrolls for both situations.
    (unfortunately it seems that other code updates have compromised their proposed patch, so I’ve left them a warning comment regarding this).

Their solution calls an interface which has access to a function that synthesizes the affect of a mouse scroll event.  So I attempt to drop the code in place… by skipping some steps and I try to call the mouse scroll function directly… failure!

There is something too different from the way that there code is received in Thunderbird on this 3 pain window!

Seeing the Forest from the Trees

Ok, so I had some of their code in place, and simply be eliminating the window._content.focus from the page didn’t solve the problem as I had hoped.  Despair started to seep in, especially as the latest firefox updates had changed the code base to my gesture support comparison and now I had to monkey around reverting my code base back to its original state with Time Machine on my mac (the backup software on my laptop).  Issue after issue arose just trying to compile and get Firefox up and running so that I could learn from it a little bit better.  I did manage to get it back and ran through a walkthrough (by leaving a trail dump(“output messages”);  as the program ran so that I could track all of the events that occurred for a single swipe!

but thought about this… maybe the interface that is being called for the scroll event has a problem with the window pane that it’s focused on.  Of course, but how to isolate out which pane I was working on…..   of course!

Javascript: The Good Parts to the Rescue!

The first thing I was to do… (in hopes that this is the correct trail that I’m following while lost in the forest) was to get some code in place that would track what the current code is actually looking at.  For those real keeners… here’s a cleaned up PDF of the output so you can follow along.

Essentially, when the scroll event happened over the message pane, it was showing up as a XPCNativeWrapper object as the event’s target object.  For the other two panes, they were coming back as XULElement object items.  hummm…. I need more information so that I can start defining what is happening.

Do to some ungrateful slams against javascript… my teacher conceded to point out the fact that we have no idea how javascipt works and that’s why we think its crap!  So he gave a very quick introduction to its advanced concepts and also pointed out two Javascript books that were the only ones worth getting.  One of which was “Javascript: The Good Parts”.  Yes, the name implies correctly that there are a number of bad parts… the beauty is this book is super concise and distinguishes the good from the bad and points out in a simple approach the more advanced (and good) parts of Javascript.  SO with this book now in hand…. I march on.

Search through the Properties

I remember in a section that I read on properties and objects that you could enumerate through the properties and he even pointed out how to avoid the properties that were functions.  This was my next step:

let doc = aEvent.target;      
var docNameX;   
for ( docNameX in doc){     
  if(typeof doc[docNameX] !== 'function') {
    dump("\n" + docNameX + " : " + doc[docNameX] + "")
dump("\n end of property list  \n\n");

This produced some wonderful output.  The XPCNativeWrapper object was rather unreliable in providing output, but the XULElements had all sorts of interesting properties related to them.  The trick now was to find some conditions that would isolate them out.

Isolating the Pains…  I mean Panes!

So I had tested some various conditions and found that even if I tested a property with the javascript object method hasOwnProperty (I also could have been using it in the wrong context) would cause the XPCNativeWrapper to crash out when it checked for a true/false against this object.  I was expecting a false scenario, but instead as mentioned, the function beyond that test just fails.  So I discovered this alternative, which successfully isolated out the pane that I was swiping over top of.

if(doc['namespaceURI'] !== null){ 
  // it is at least a XULElement (folder pane or message thread page 
  if(doc['tooltip'] == "folderpopup"){ 
    dump("\n  You are over the Folder Pane!"); 
  } else { 
    dump("\n  You are over the Message Thread Pane!"); 
} else { // its a XPCNativeWrapper (message viewer) 
  dump("\n  You are over the Message Pane!"); 
dump("\n  Did you find your correct Pain!  \n");


Next Steps….  How to trigger an event for these conditions based on the correct pane now being capable of isolation.




Posted in Open Source, _Old Archived Posts | Leave a comment

Back to the Javascript, Part 2

Ok, so this is a followup blog to a previous one where I made some fun at the Javascript language.  Criticizing it as a toy and possibly in my own mind, being critical for its cryptic looking nature.  And thus, I was attacked by my instructor for my comments, I was thrown to the ground, my face mashed in the mud… I was humbled actually.  Ok, so he didn’t attack me… we had a couple pleasant jabs at each other.  But what came from it was some inspiration for him to introduce our Open Source Development class to some of the better features of Javascript and an exploratory environment where we got to discuss some of its very good and complex features.

I think the “Back to the Future II” spin on things is appropriate here.  We have a language that could be much better… it’s almost like the discovery of what it can do, based on the good qualities of its precursor languages (scheme,lisp) leaves it to have some very special abilities.  But where is our Marty McFly to save us by going back to the future to protect a language that could have been great.  Instead Javascript lands upon us as if Biff Tannen had butchered it by poisoning the past.

Douglas Crockford

This name should be known by anyone who has the need to be a Javascript developer.  He is the mind behind JSON, the data-interchange format to replace XML, he also developed, in some of his off time, JSLint which is a javascript code quality tool (highly recommended by Crockford… even for himself) and author of a very very cool book, Javascript: The Good Parts.  I did purchase this book, and not only is it nice due to the fact that its barely 150 pages, but its cram packed full of good stuff, and the fluff is pretty much removed so it gets right to the guts of it.  Our lab was to review this video of Douglas doing a talk on “The Good Parts”, but I guess I purchased the book first… then listened to the review.

Brief Conclusion to a Brief Blog

I’m glad I slammed Javascript in order to provoke my instructor.  It’s brought out all sorts of great learning on javascript.  Personally, I don’t like not knowing how to do things.  And for whatever reason, most people (even authors of Javascript books) haven’t really got to know Javascript the way that Crockford has… and its great to have an instructor that has learned from his material.  Douglas Crockford points out extensively how Javscript is used wrong and how much of its potential is hidden from people.  His book is a great eye opener for those that are interested in any sort of web development or development where Javascript excels!  I know this sounds twisted….

I’m very excited about getting to know Javascript!   (the right way)

Posted in Open Source, _Old Archived Posts | Leave a comment