IE Automation Library and Web Macro Recorder

IE9 Automation and Web Macros

2011-05-12T01:27:00.000-07:00

Twebst Automation Studio 2.6 - New Release!

I am once again happy to announce a new version of Twebst Automation Studio. What's new:

IE9 web automation and macro support.
custom DLL injection instead of BHO browser extension (which in IE9 generates security warnings).
TwebstBHOps.dll is no longer needed.
FindBrowser and FindAllBrowsers now work with any application that embeds Internet Explorer web control.
Bug fixes and enhancements.

Download Twebst 2.6

IE8 close tab programmatically

2010-07-08T01:00:00.000-07:00

Sometimes you may wish to close a browser instance and Twebst Library has support for that. You just have to call Close method against the Browser web automation object you want to close.

Let's see a short JScript web automation macro that closes the first IE8 tab loaded from google.com domain. If the closing tab is the last tab inside the IE frame then the whole browser will be closed.

// Create core object.
var core = new ActiveXObject("Twebst.Core");

// Find a browser for which the displayed URL contains "google.com".
core.useRegExp = true;
var b = core.FindBrowser("url=.*google.com");
b.Close();

There are more Twebst methods for automating IE7 and IE8 tabs that I'll talk about in future articles: OpenNewTab, activeTab, isTabActive, tabList

Twebst Automation Studio 2.5 - New Release!

2010-07-07T09:30:00.000-07:00

This blog was too quiet for some time but after another development cycle I am happy to announce a new release: Twebst Automation Studio version 2.5

New features:

ClosePopup web automation method
ClosePrompt web automation method
closeBrowserPopups web automation property
OpenNewTab web automation method
tabList web automation property
activeTab web automation property
isTabActive web automation property
asyncHtmlEvents web automation property
RightClick support added in web recorder
index attribute added to FindFrame method
minor bug fixes in web automation library, web recorder and help system

Download Twebst - IE Automation Software

Move data from Excel to IE and back with VBA web macros

2010-03-07T07:42:00.000-08:00

Let's continue my previous post (IE8 Automation using Excel VBA Macro) with some VBA Internet Explorer automation code:

' Don't forget to add reference to TwebstLib from menu Tools/References
Dim core As TwebstLib.ICoreVba
Set core = New TwebstLib.core

Dim browser As TwebstLib.IBrowserVba
Set browser = core.StartBrowserVba("http://www.google.com/")

For i = 1 To 5
   ' Get the current text to be translated
   Dim textToTranslate As String
   textToTranslate = ActiveSheet.Cells(i, 1).Formula

   ' Twebst web automation part
   Call browser.Navigate("http://www.google.com/language_tools?hl=en")
   Call browser.FindElementVba("textarea", "id=source").InputText(textToTranslate)
   Call browser.FindElementVba("select", "name=sl").Select("English")
   Call browser.FindElementVba("select", "name=tl").Select("French")
   Call browser.FindElementVba("input submit", "text=Translate").Click

   ' Find result span object
   Dim resultObj As TwebstLib.IElementVba
   Set resultObj = browser.FindElementVba("span", "id=result_box")

   ' Get the text result
   Dim translatedText As String
   translatedText = resultObj.Text

   ' Save the result into Excel sheet
   ActiveSheet.Cells(i, 2).Formula = translatedText
Next

Points of interest:

in VBA web macro new methods are used (see FindElementVba). The newly added methods (from version 2.1) are especially designed for VBA language and they work the same as they non-VBA counterparts except the fact no variable number of search conditions are supported.
Twebst web recorder was updated to automatically generate VBA macro code against the newly added methods.
all web page loading synchronization is provided by default by the library.
when automating web controls all necessary HTML events are fired up against target object (see fireEvent) to simulate user's web action.

Downloads:

Twebst Automation Studio PRO you'll need this installed before running the sample VBA macro
TwebstVBA.zip containing sample Excel and BAS files.

IE8 Automation using Excel VBA Macro

2010-03-03T05:18:00.000-08:00

In my previous posts (Excel and IE automation - the WSH way, Extract web data into Excel.Application with browser macro) I automated both IE and Excel using WSH. Now I'm going to automate Internet Explorer directly from an Excel macro using the VBA language.

The VBA macro will work against a sheet containing phrases to be translated using Google Translate tool. It automatically gets each phrase, opens IE browser, navigates to translation page, waits the web page to complete loading, fills out the text to be translated into HTML web controls, press submit button, gets back the result and saves it into the same Excel sheet.

It is possible to directly automate IE browser from VBA macro but it is not always as simple as it should be. (see my older post: What's wrong with Internet Explorer Automation?) Using Twebst Automation Studio when automating IE from VBA has some advangates:

it works with all Internet Explorer versions: IE6, IE7, IE8 and with IE protected mode.

it works on all Windows versions: XP, Vista, Win7

easily search of elements across all sub-documents inside frames/iframes hierarchy

search HTML elements using regular expressions

simulate user actions on all HTML controls (button, combo-box, list-box, edit-box, upload control) and fire all necessary HTML events

auto-generate VBA code with powerful web recorder

much more (see Twebst Automation Studio website)

That's all for now. Until our next encounter, check the download section below and have fun with the Excel VBA macro code.

Downloads:

Twebst Automation Studio PRO this web automation tool is used in VBA maro so you have to install it first.

TwebstVBA.zip containing sample XLS and BAS files.

Extract web data into Excel.Application with browser macro

2010-02-25T04:44:00.000-08:00

Let's continue my previous post where I presented a browser macro that automatically translates phrases in an Excel document using Google Translate tool. As you cand see in code, the WSH script performs the following steps:

instantiate "Twebst.Core" web automation object and start an IE browser.
instantiate "Excel.Application" automation object and open XLS document.
get a phrase to be translated from Excel document.
navigate to Google translate tool web page.
fill out web controls (the text to be translated, source and target language combo-boxes) and then automatically press submit button.
wait the web page to load, get the translated text and save it into Excel document.

Let's see some code. I will skip the initial setup phase and go directly to web automation part which is more interesting:

for (var i = 1; i <= 5; ++i)
{
 // Get current text to translate from XLS document.
 var textToTranslate = excelApp.ActiveSheet.Cells(i, 1).Formula;

 // Web automation part implemented with Twebst Automatio Studio.
 browser.Navigate('http://www.google.com/language_tools?hl=en');
 browser.FindElement('textarea', 'id=source').InputText(textToTranslate);
 browser.FindElement('select', 'name=sl').Select('English');
 browser.FindElement('select', 'name=tl').Select('French');
 browser.FindElement('input submit', 'text=Translate').Click();

 // Get the reszult and save it into XLS document.
 var translatedText = browser.FindElement('span', 'id=result_box').text;
 excelApp.ActiveSheet.Cells(i, 2).Formula = translatedText;
}

Points of interest:

FindElement method waits for the IE web page to be completely loaded before searching the element. The timeout value can be changed programatically (see loadTimeout, searchTimeout properties).
InputText method sets the specified value in edit/password web controls and fires all necessary HTML events (onkeydown, onkeypress, onkeyup, onchange, etc).
Select method fully automates combo-boxes and list-boxes web control and fires all necessary HTML events (like onchange).

Downloads:

Twebst Automation Studio - you need to install this web automation tool before running the demo WSH script below.
ExcelToWebMacro.zip - web scraping demo script that saves results into Excel file.

Excel and IE automation - the WSH way

2010-02-24T01:32:00.000-08:00

I can think of two possible scenarios when automating Excel and Internet Explorer makes any sense:

you have data stored in Excel sheets and you want to push data into online web forms.

you need to extract data from web and save them into Excel xls file.

There are two ways to implement this:

automate Internet Explorer browser from Excel VBA macro.

automate both IE and Excel from an external process using OLE automation.

Today I will talk about the second option; I will automate both Internet Explorer and Excel application from a WSH script using Twebst Automation Studio. Here's a diagram showing the work-flow:

The WSH web macro in this tutorial opens an XLS file containing phrases to be translated using Google Translate tool. It then automatically gets each phrase, opens an IE browser, navigates to translation page, fills out the text to be translated, press submit button, gets back the result and saves it into the same XLS file.

That's all for now! (as I try to keep articles relatively short) Meanwhile, you should download Twebst Automation Studio and the web macro sample to see it at work. Then, you can continue with the next article explaining the web macro code and comments: Extract web data into Excel.Application with browser macro

Downloads:

ExcelToWebMacro.zip

Twebst Automation Studio (FREE edition available btw)

IE8 automation with Excel VBA macro

2010-02-05T00:58:00.000-08:00

Twebst Automation Studio Ver 2.1 New Release!

Many people requested for VBA (Excel, Word) direct web automation support in Twebst Library. The previous version could not be easily used from VBA because of the way Excel works with methods with variable number of parameters (which is the case for many Twebst methods like: FindElement, FindBrowser, etc).

The new Twebst 2.1 version includes a new full set of interfaces and methods specially designed for Excel VBA. The web recorder was also updated to generate VBA code that you can easily include in your Excel macro. To see what's new in Twebst ver 2.1 check changelog.txt

Automating Internet Explorer from Excel VBA macro is now possible with Twebst Automation Studio 2.1.

Get It While it's Hot!

.

How to automate existing instances of Internet Explorer - Tutorial(4)

2009-12-21T00:03:00.000-08:00

As explained in a previous tutorial, a web macro can start by connecting to an already running instance of Internet Explorer browser. A web macro usually starts a new Internet Explorer page and then navigates to a URL to begin with. There are times when automating an existing IE window is a more natural alternative. Let's start with a JScript/WSH simple web macro:

// Create core object.
var core = new ActiveXObject("Twebst.Core");

// Find a browser for which the displayed URL contains "yahoo".
core.useRegExp = true;
var b1 = core.FindBrowser("url=.*yahoo.*");
b1.Navigate("www.google.com");

// Find a browser for which the title is "Google" (exact match).
core.useRegExp = false;
var b2 = core.FindBrowser("title=Google");

b2.Close();

Basically FindBrowser method searches for a browser page based on TITLE and/or URL. If core.useRegExp is false then an exact search is used. If core.useRegExp is true then the search uses regular expressions. Once connected to a IE browser instance, you get a Browser object to play with.

Of course, this works for IE6, IE7, IE8 on Windows XP, Vista and Windows 7 (32 and 64 bit) and it's compatible with Internet Explorer Protected Mode.

What can you do with a Browser object? Quite a lot of things: Navigate to a URL, FindElement and FindFrame inside it, Close the browser, get information about the browser, automate modal and modeless HTML dialgos and wait to complete navigation and loading.

Enough talk! It's time to Download Twebst Automation Studio and see it for yourself.

IE macro - how to get programmatic control over HTML elements - Tutorial(3)

2009-12-14T08:01:00.000-08:00

Finding elements inside HTML frames/iframes hierarchy with Twebst Web Automation Library

Once you have started a web macro and you've got a Browser object to play with, it's time to get programmatic access HTML controls inside the web page. As any web developer should know, you can easily find elements inside a document using document.getElementById.

In the world of web automation things are a bit different; web macros need to access HTML elements in any web document and at any level of DOM hierarchy. The target HTML element can be inside frames/iframes loaded from various domains, subject of cross-frame scripting security restrictions (see my older posts: "When IHTMLWindow2::get_document returns E_ACCESSDENIED" and "When IHTMLWindow2.document throws UnauthorizedAccessException").

Here's a simple VBscript web macro that automates a Google search in Internet Explorer:

Dim core
Dim browser
Set core    = CreateObject("Twebst.Core")
Set browser = core.StartBrowser("http://www.google.com/")

Call browser.FindElement("input text",   "name=q").InputText("codecentrix")
Call browser.FindElement("input submit", "name=btnG").Click()

What is so great about the code above is the way FindElement method works: first it waits for the browser to load the HTML document then it searches for the element thru all frame/iframe hierarchy.

Once an Element is found you can perform actions on it like: Click, RightClick, InputText, Check, Uncheck, Select, Highlight etc. What is even cooler than this, is that you don't have to write these statements by yourself. They will be automatically generated by Twebst Web Recorder which you can get for free:

Get Free Web Recorder for Internet Explorer

IE8 automation: How to programatically open an URL - Tutorial (2)

2009-12-09T00:11:00.000-08:00

Get started with web macros in IE

Every web macro has to start somewhere, has to start somehow. There are basically two scenarios of automating Internet Explorer browser:

start a new IE browser and navigate to a URL to begin with

connect to an existing instance of Internet Explorer browser and continue automation

In this short tutorial I'll show you how to open a new Internet Explorer browser and open an URL with Twebst Automation Studio. Let's start with a short JScript web macro.

var core    = new ActiveXObject('Twebst.Core');
var browser = core.StartBrowser('http://www.google.com/');

The code is quite self-explaining. It creates a Twebst object and then opens a given URL in a new IE browser instance. What you get back from StartBrowser call is a Browser object that can be used to further automate Internet Explorer.

What can you do with a browser object? Quite a lot of things: navigate to a URL, find HTML elements and frames inside it, close the browser, get information about the browser, automate modal and modeless HTML dialgos and wait to complete navigation and loading.

That's all for now, but these feature will be covered in next tutorials!

Get Twebst Automation Studio Free Edition

(Automation Library and Macro Recorder for Internet Explorer)

Automate HTML File Upload Control - Tutorial (1)

2009-11-08T17:45:00.000-08:00

If you ever tried to programatically change the value of a HTML File Upload Control you probably noticed that <input type="file" /> element is read-only! This happens for good security reasons: web pages scripts should not be able to upload random files without user consent. Unfortunatelly the control is read-only for browser extensions and add-ons too (BHO, toolbars, side bars, etc).

For IE6 and IE7 the upload control may be automated by setting the focus to "Internet Explorer_Server" window and generating Win32 keyboard events. For IE8, the upload control is read-only, the user can only set a value by pressing "Browse" button and choosing a file.

With Twebst Automation Studio, automating HTML File Upload Control can't be easier. Here's a short VBScript sample of doing it:


Option Explicit
Dim core
Dim browser
Set core = CreateObject("Twebst.Core")
Set browser = core.StartBrowser("http://www.2shared.com/")

Call browser.FindElement("input file", "id=upField").InputText("C:\somepath\photo1.jpg")

Get Twebst Automation Studio Free Edition!

New Web Macro Recorder released!

2009-11-03T09:19:00.000-08:00

After several months of complete radio silence, things have been moving forward for Twebst Library which turned into a more powerful and mature product.
Please welcome Twebst Web Automation Studio!

Twebst Automation Studio is an advanced web automation framework for Internet Explorer that can be used within any environment that supports COM, from scripting languages (JScript, VBScript) to high level programming languages (C#, VB.Net, C++).

The framework includes two components: Twebst Web Recorder that automatically generates most of the automation code and Twebst Library which is a collection of programable objects that creates a web automation API.

But one image is better than thousand words...

Twebst Web Recorder features:

Easily create web macros through an intuitive graphical interface.
Generate web automation code using the language of your choice: JScript, VBScript, C#, VB.Net, C++
Record web actions on all HTML controls (button, combo-box, list-box, edit-box)

And one more thing: FREE version available!

Download Twebst Automation Studio

Web Data Extraction with C++ Web Macro

2009-05-26T20:05:00.000-07:00

Web data extraction or web scraping can be implemented in various ways. Today I will use Twebst Web Automation Library to extract search results from Google using DOM parsing method and Internet Explorer automation (you need to install Twebst Library first).

Here are the steps that C++ web macro will perform in order to extract results from Google search:

Open an Internet Explorer browser and navigate to Google site.
Find the search edit box and fill out the word to search.
Find the submit button and click it.
Wait until the page is loaded and find a DIV with id=res
Find the collection of all H3 elements inside the DIV element.
Extract the text and URL and display it.

Enough talk! Let the code speak for itself.

// Start a new Internet Explorer instance and navigate to a given URL.
IBrowserPtr pBrowser = pCore->StartBrowser("http://www.google.com/");

// Find search edit box in page and type some text into it.
IElementPtr pSearchEdit = pBrowser->FindElement("input text", SearchCondition("name=q"));
pSearchEdit->InputText("codecentrix");

// Find search button and click it.
IElementPtr pSearchBtn = pBrowser->FindElement("input submit", SearchCondition("text=Google Search"));
pSearchBtn->Click();

// Find the DIV element where the result are displayed.
IElementPtr pResultDiv = pBrowser->FindElement("div", SearchCondition("id=res"));

// Get all found results and print them in console.
IElementListPtr pResultList = pResultDiv->FindAllElements("h3", SearchCondition());

// Display only the header result (text and url).
for (int i = 0; i < pResultList->length; ++i)
{
    // Get current H3 in the list.
    IElementPtr pCrntResult = pResultList->Getitem(i);

    // Find first and only anchor inside H3
    IElementPtr pCrntAnchor = pCrntResult->FindElement("a", SearchCondition());
    CComQIPtr<IHTMLAnchorElement> spCrntAnchor = pCrntAnchor->nativeElement;

    // Get URL from IHTMLAnchorElement.
    CComBSTR bstrURL = "";
    spCrntAnchor->get_href(&bstrURL);

    // Display results.
    wcout << pCrntResult->text << L"\n" << bstrURL.m_str << L"\n\n";
}

Download:

CppMacro.zip (full source code and exe)
Twebst Web Automation Library

IE Web Login Automation

2009-05-19T21:03:00.000-07:00

One highly repetitive web task is the logon to a web site. This is a common scenario where Twebst Web Automation Library really shines. Here is a short web macro written in JScript language that automatically logs you on Yahoo Mail site. All you have to do is to replace "UUUUUUUUUU" and "PPPPPPPPPP" with your user name and password in the code below.


// Open a browser and navigate to yahoo mail login page.
var core = new ActiveXObject("Twebst.Core");
var browser = core.StartBrowser("https://login.yahoo.com/config/mail?.intl=us");

// Find login fields.
var u = browser.FindElement("input text");
var p = browser.FindElement("input password");
var s = browser.FindElement("input submit");

// Log on to site by filling the user-name and password fileds and then click submit boutton.
u.InputText("UUUUUUUUUU");
p.InputText("PPPPPPPPPP");
s.Click();

FindElement searches thru all frames/iframes hierarchy for the first input element of type text/password/submit. Additional conditions can be specified for search (like searching an element by id/name or any other HTML attribute). Search conditions can make use of regular expressions if needed.

One more important thing is that FindElement method waits for the web page to be completely loaded before searching the element (the timeout can be specified by using core.loadTimeout property). Read more about Twebst Library...

Download:

Twebst Library Pro (it must be installed for the macro to work).

YahooMail.js

Script Of The Day system if you want to easily install more login templates.

Twebst Web Automation Library v1.40 released

2009-05-18T20:05:00.000-07:00

Twebst version 1.40 is launched!
Main changes include IE8 compatibility, better support for working with embeded IE browser control, support for modal and modeless HTML dialogs and functions for clipboard access.

Here is the list of new features and enhancements:
- NEW: IE8 is now supported
- ENH: core.AttachToNative* methods work now with hosted IE browser control
- BUG: various fixes
- NEW: core.foregroundBrowser property
- NEW: core.productName property
- NEW: core.productVersion property
- NEW: core.GetClipboardText method
- NEW: core.SetClipboardText method
- NEW: core.AttachToWnd method
- NEW: core.NativeWindowToNativeBrowser method
- NEW: core.NativeWindowToNativeDocument method
- NEW: core.NativeWindowToNativeDocument
- NEW: browser.FindModalHtmlDialog method
- NEW: browser.FindModelessHtmlDialog method
- NEW: element.GetAttribute method
- NEW: element.SetAttribute method
- NEW: element.RemoveAttribute method
- NEW: element.tagName property
- NEW: element.FindParentElement method
- NEW: core.RightClick method-
- Find more ...

Free Download Twebst Library 1.40

Homemade Handcrafted Help System

2009-04-29T11:15:00.000-07:00

A good documentation is very important for any serious project. It comes a moment in life, when programmers find themselves working on the help system. That is what happened to me during the later stages of Twebst Web Automation Library project. Even though this project is not open source, I try to make public as many parts of it as possible. Today I will present Twebst Help System and how this tedious and annoying task of creating it, was automated.

Twebst is a library of COM objects used to automate Internet Explorer browser. The objects and the supported properties and methods have to be documented . The page structure is the same for every object/method/property and it also contains code samples that need syntax highlighting. This is good news because it leaves a lot of place for templates and automation.

Here is the solution:

The template is an XML document. When documenting an object/method or property the focus is on the content rather than on formatting the text. There is one XML file for each object/method/property.
A WSH script written in jscript parses the XML document and adds syntax highlighting to sample code in the documentation page. Regular expression are used for parsing.
cross references are added automatically by the same script.
then a XSL transformation is applied to convert XML source to a HTML document that will be eventually written to disk.
The whole process is optimized by removing unnecessary operations like generating the HTML when it already exists and is newer than its XML source.
Finally the HTML documents refers a CSS style sheet to easily change the look.

It goes like this:
XML + JScript-> XML with color syntax and cross references + XSL -> HTML + CSS -> CHM

For local help, the CHM compiler is invoked as a final step and a CHM Help File is generated. All you have to do is launching Build.js script you may find in the archive below.

Downloads: TwebstHelp.zip

Prerequisites: In order to build the CHM file you'll need HTML Help Workshop from Microsoft.

SQL CE 3.5 SP1 and MSVCR80

2009-03-18T11:18:00.000-07:00

I am involved in Web Replay (the best password manager on the market) development from its inception. Recently, we have upgraded SQL CE engine to the latest 3.5 SP1 version. Web Replay is built against Visual C++ runtime version 9 which is deployed by including Microsoft Visual C++ 2008 Redistributable Package (x86) in the setup. But that was not enough because Microsoft SQL CE 3.5 SP1 depends on Visual C++ runtime version 8.0 which is NOT included in the redistributable kit!

If you encounter this problem you have no option but to include vc2005 redist in your setup. You could think that giving up the upgrade to 3.5 version and continuing using SQL CE 3.1 is an option. Well, it isn't! SQL CE 3.1 is crashing randomly on Windows XP SP3 (even Microsoft SQL Server Management Studio is reporting some memory corruption error before completely freezing).

RegisterHotKey in C#

2009-02-20T06:43:00.000-08:00

I like .Net framework for its huge class library covering almost anything you'll ever need. One thing I could not find was registering a windows hot-key in C#. It seems the framework does not provide a solution for system-wide hot keys. So I had to use the good old RegisterHotKey Win32 function.

To call it from C#, you need to use P/Invoke service to invoke the unmanaged function residing in user32.dll


using System.Runtime.InteropServices;

[DllImport("user32.dll")]
private static extern bool RegisterHotKey(IntPtr hWnd, int id, int fsModifiers, int vk);

The hWnd parameter is the handle of a form that will receive WM_HOTKEY message. To get the handle of a Form simply use Handle property. In order to process WM_HOTKEY message, your form needs to override WndProc method.

Just one more thing you should know: RegisterHotKey and ShowInTaskBar property don't mix well. It seems that a new window is created each time ShowInTaskBar property is called so your WndProc mehod will stop recieving hot-keys. You need to RegisterHotKey again because Handle property returns a new window handle after ShowInTaskBar was called.

WSH and clipboard access

2009-01-05T02:37:00.000-08:00

I did some Windows Script Host programming recently and I was pleasantly surprised by its power, features and flexibility. One thing that I couldn't accomplish was accessing the clipboard from WSH. Digging the internet I found some solutions like this one based on Internet Explorer Automation. There are several problems with this approach as you can read in my article about Internet Explorer Automation: What's wrong with Internet Explorer Automation?

My solution for scripting the clipboard content in WSH is a regular COM object created with VC++ and ATL.

Download full source code and compiled DLL: WSH_clipboard.zip
To install the COM object run register.bat

I found scripting the clipboard useful enough to add this feature to the next release of Twebst Web Automation Library.

Free Web Macros for Internet Explorer

2008-12-15T19:09:00.000-08:00

As I presented in my previous post, automating Internet Explorer can be a difficult task.
Twebst Web Automation Library can make things easier.

It gives full programmatic control over the Internet Explorer browser. Twebst is a library of COM object that can be used within any environment that supports COM, from scripting languages (JScript, VB Script) to high level programming languages (C#, C++). For more information, see Twebst Libray Online Documentation. And yes, it's free!

Get it FREE!

What Twebst can do?

increase productivity by automating repetitive web tasks
automate regression testing of web applications
automate web actions and data-entry
automatically log in to different web sites
fill out web-forms automatically
extract data from web pages (web scraping).
monitor web pages

Twebst features

Start new browsers and navigate to a specified URL.
Connect to existing browsers.
Search and access HTML elements and frames inside browsers.
Intuitive names for HTML elements using the text that appears on the screen.
Advanced search of browsers and HTML elements using regular expressions.
Perform actions on all HTML controls (button, combo-box, list-box, edit-box etc).
Simulates user behavior generating hardware or browser events.
Get access to native interfaces exposed by Internet Explorer so you don't need to learn new things if you already know IE web programming.
Synchronize web actions and navigation by waiting the page to complete in a specified timeout.
Available from any programming or script language that supports COM
Optimized search methods and collections.

What's wrong with Internet Explorer Automation?

2008-12-10T09:54:00.000-08:00

The Microsoft Office products (Word, Excel, Power Point, Access, Outlook) allow their users to manipulate Office documents from Visual Basic or Visual Basic for Applications (VBA) code. It is possible to write a VBA macro in Excel that initializes a series of cells, and uses the cells to display a chart for instance.

Automation is the process of controlling one product from another product with the result that the client product can use the objects, methods, and properties of the server product. The client has access to the object model of the server.

Though Internet Explorer browser is not part of the Office suite, it supports automation. Here is a short sample:

// Create an IE automation object.
var ie = new ActiveXObject("InternetExplorer.Application");

// Make it visible and navigate to a given URL.
ie.Visible = true;
ie.Navigate("http://www.google.com/");

// Give it some time to load the page and then get the document.
WScript.Sleep(3000);
var doc = ie.Document;

// Fill out search field.
var edit = doc.getElementsByName("q").item(0);
edit.value = "codecentrix";

// ... and press the submit button.
var submit = doc.getElementsByName("btnG").item(0);
submit.click();

Here is ie_auto.js file for download.
However there are problems with Internet Explorer automation:

it may not work at all on Windows Vista unless the script is running at the same integrity level as iexplore.exe process. Simply clicking the js file won't do it. The script will run at medium integrity level and Internet Explorer has low integrity level and as result the script fails. If you run the script at high integrity level the newly started IE instance will have the same high integrity level and the script works (but this is not the best option from a security point of view). Changing the integrity level of the running script (or application) is not always the most desirable or easiest thing to do.

no support to "connect" to already existing IE documents.

sub-documents in different domains are not accessible for scripting due to cross scripting security issues (see my older posts: "When IHTMLWindow2::get_document returns E_ACCESSDENIED" and "When IHTMLWindow2.document throws UnauthorizedAccessException").

difficult search of elements across all sub-documents inside frames/iframes (and sometimes impossible, see the point above).

difficult and time consuming search of HTML elements on attributes other than id or name (getElementById and getElementsByName are the only methods I know that search elements directly wihtout browsing element collections which might be very slow when performed out of process).

no tab support (only launch a new tab).

no direct support for synchronizing input actions (clicks, keys) with the HTML document loading (it could be implemented by registering to IE events like document complete or looping while the browser becomes ready to accept inputs).

no advanced search criteria like regular expression or searching on multiple attributes.

If you are interested in solving the issues above, let me introduce a project I've been working on for some time now. Here's Twebst, web automation library for Internet Explorer!

Get it FREE!

(to be continued)

Creating shortcuts to Quick Launch Toolbar with WSH

2008-11-25T16:20:00.000-08:00

I had this problem of creating shortcuts to Quick Launch Tollbar while working on Script Of The Day application. This small product is almost entirely created using JScript and Windows Scripting Host (WSH).

SpecialFolders method of WScript.Shell object provides the full path for some special folders like Desktop and Favorites but the directory for Quick Launch Toolbar is not supported. To get it I used %userprofile% env var like this:

var shell          = WScript.CreateObject("WScript.Shell");
var quickLaunchDir = shell.ExpandEnvironmentStrings("%userprofile%") +
                     "\\Application Data\\Microsoft\\Internet Explorer\\Quick Launch";
var oShellLink     = shell.CreateShortcut(quickLaunchDir + "\\Codecentrix.lnk");

oShellLink.TargetPath   = "http://www.codecentrix.com/";
oShellLink.IconLocation = "http://www.codecentrix.com/favicon.ico";
oShellLink.WindowStyle  = 1;
oShellLink.Description  = "Web Site";
oShellLink.Save();

Downloads:

QuickLaunch.js sample file
Script Of The Day application (FREE)

focus vs fireEvent("onfocus")

2008-08-09T10:18:00.000-07:00

While working on Twebst web automation library I encountered this problem: how to simulate setting the focus on HTML edit controls in Internet Explorer? There are two ways to do this.

Call IHTMLElement2::focus() method on target element that "causes the element to receive the focus and executes the code specified by the onfocus event".
Rise onfocus event on target element by calling IHTMLElement3::fireEvent() method.

The two approaches are quite similar but there are some interesting differences.

fireEvent("onfocus") does not actually set the focus on the element, it just executes the code of the onfocus handler event.
Calling focus method sets the focus on target element and call the onfocus event handler but not immediately. The onfocus event seems to be inserted in a queue and its handler is executed asynchronously after the current handler is finished.
If focus method is called from inside the onfocus handler nothing happens if the control already has the focus (that prevents an infinite recursion).

Example:

<html> <script type="text/javascript" language="javascript"> function BtnFocusClick() { document.getElementById('editTest').focus(); window.status += "b"; } function BtnOnFocusClick() { document.getElementById('editTest').fireEvent('onfocus'); window.status += "c"; } function EditOnFocus() { window.status += "a"; } </script> <body> <input type="text" onfocus="EditOnFocus()"; id="editTest"/><br/> <input type="button" value="focus" id="btnFocus" onclick="BtnFocusClick();"/> <input type="button" value="fire onfocus" id="btnOnFocus" onclick="BtnOnFocusClick();"/> </body> </html>

If pressing the button "fire onfocus" button the message in the Internet Explorer status bar is the expected one "ac". If pressing the "focus" button, the message is in reverse order than expected: "ba". That suggests that EditOnFocus handler is called after BtnFocusClick exit.

IHTMLDocument3::getElementsByTagName and IHTMLElementCollection

2008-06-19T02:22:00.000-07:00

A common task when writing Internet Explorer extensions is to browse a collection of objects based on a specified element tag-name. To work with collections IE provides IHTMLElementCollection interface that represents a collection of elements in an HTML document.

Usually a collection is retrieved by calling methods of IHTMLDocument2 interface. For some tag-names there specialized methods to retrieve collection (IHTMLDocument2::get_anchors , IHTMLDocument2::get_applets, IHTMLDocument2::get_forms, IHTMLDocument2::get_images, IHTMLDocument2::get_links, IHTMLDocument2::get_scripts).

To get all elements collection there is IHTMLDocument2::get_all.
One way to get a collection of elements having a specified tag-name is:

// CComQIPt<IHTMLDocument2> spDocument is a document object.

CComQIPtr<IHTMLElementCollection> spAllCollection;
HRESULT hRes = spDocument->get_all(&spAllCollection);
_ASSERTE(SUCCEDED(hRes) && (spAllCollection != NULL));

// Get the sub-collection of elements that have the "input" tag name.
CComVariant varTagName(CComBSTR("input"));
CComQIPtr<IDispatch> spDispCollection;
hRes = spAllCollection->tags(varTagName, &spDispCollection);

CComQIPtr<IHTMLElementCollection> spInputCollection = spDispCollection;
_ASSERTE(spInputCollection != NULL);

// Now you can browse spInputCollection using
// IHTMLElementCollection::item and IHTMLElementCollection::get_length methods.

The second method is:

// CComQIPt spDocument is a document object.
// Query for IHTMLDocument3 interface
CComQIPtr<IHTMLDocument3> spDoc3 = spDocument;
_ASSERTE(spDoc3 != NULL);

// Get the collection of elements that have the "input" tag name.
CComQIPtr<IHTMLElementCollection> spInputCollection;
HRESULT hRes = spDoc3->getElementsByTagName(CComBSTR("input"), &spInputCollection);
_ASSERTE(SUCCEDEED(hRes));

// Now you can browse spInputCollection using
// IHTMLElementCollection::item and IHTMLElementCollection::get_length methods.

ATL thunks and Windows DEP story

2008-05-19T14:00:00.000-07:00

When using older ATL versions the program may generate an access violation due to Data Execution Prevention. This is basically a memory protection feature that prevents executing code from memory pages marked as non-executable.

The ATL implementation of CWindow class uses a technique called thunk. A thunk is a small piece of code that ATL generates in a region of memory allocated on the heap. Older versions of ATL did not set the execution flag on the allocated memory pages and that generates a crash on Windows machines where DEP is enabled.

Usually ATL is used in Internet Explorer extensions. If DEP is enabled then the result is a browser crash. This is a good reason to upgrade to Visual Studio 2005 or later. Find more on how to activate DEP in IE7 here.

ATL + STL = CAdapt

2008-04-21T02:36:00.000-07:00

I didn't know about CAdapt class until I tried to convert a VS2003 project to VS2005. Everything worked OK but I couldn't compile it! The compiler complained about:

std::list<CComQIPtr<IHTMLElement> >.

I found that what was accepted by VC++ 2003 compiler is not accepted by VC++ 2005 compiler. The reason of for is the address operator overloaded by CComQIPtr class. std::list class needs the address of a CComQIPtr object but & operator returns a IHTMLElement* address.

Here's where CAdapt class comes to save us. The list of smart pointers becomes: std::list<CAdapt<CComQIPtr<IHTMLElement> > >
Also you need to use m_T member where needed.

Did you know that ? fatal error C1091: compiler limit

2008-04-15T04:17:00.001-07:00

It doesn't happen every day to find a new compiler error message. Here's one I didn't expect:

fatal error C1091: compiler limit: string exceeds 65535 bytes in length

I was about to complete my programming task when this error struck and I really needed a very long string. On VC++ 2003 compiler the limit is even smaller, 16 Kb.

The conclusion of this story: don't put the whole story of your life inside a C++ string constant!

.Net and COM interop story

2008-03-31T01:16:00.000-07:00

.Net allows programmers to reuse COM components in their managed code. To make this possible a managed wrapper object around the native object is needed. Besides that, one can use the COM object like any other managed object. Even if it sounds simple, you have to be aware of the differences between the CLR's object lifetime management and the COM version of object lifetime management.

COM programmers have to call Release on every interface that has been AddRef'ed. For C# programmers using COM objects that means AddRef is called when:
- a COM object is created.
- a COM object is returned by calling a method or a property.
- a COM object is cast'ed to another COM interface type.

To release a COM object in C# there are two options:
- leave the GC to collect managed wrappers and to call their finalizers that will call Release on native COM object.
- manually call Marshal.ReleaseComObject on every interface used in the code.

Let's see a short example using COM objects exposed by IE. The code bellow changes the color of every link in a HTML document.


// IHTMLDocument2 doc;
foreach (IHTMLElement elem in doc.all)
{
    IHTMLAnchorElement anchor = elem as IHTMLAnchorElement;
    if (anchor != null)
    {
        elem.style.color = "red";
    }
}

This first approach leaves the task of releasing COM objects to garbage collector. Let's manually release COM objects now:

// IHTMLDocument2 doc;
IHTMLElementCollection allCollection = doc.all;
foreach (IHTMLElement crntElem in allCollection)
{
   IHTMLAnchorElement anchor = crntElem as IHTMLAnchorElement;
   if (anchor != null)
   {
       IHTMLStyle style = crntElem.style;
       style.color = "red";

       Marshal.ReleaseComObject(style);
       Marshal.ReleaseComObject(anchor);
   }

   Marshal.ReleaseComObject(crntElem);
}

Marshal.ReleaseComObject(allCollection);

As you can see the number of code lines doubles! I personally prefer to leave the task of releasing COM objects to GC even if they will be eventually released after some time when GC comes into action.

Some might be tempted to call GC.Collect after a large chunk of code that work with COM objects but this could be even worse because other managed objects could be promoted to next GC generation and their lifespan is therefore longer than necessary.

In theory it is possible to create a lot of large COM objects that will exceed the native heap while the managed heap has a lot of available memory because managed wrappers are smaller in size. GC won't be called in this scenario so the native heap won't be freed.

If your application suffers from this kind of memory allocation problem, maybe using COM objects from managed code is not the best approach for you.

The game of programming. Programming the game.

2008-02-25T20:07:00.000-08:00

The first computer I've ever seen was a Z80 Spectrum. That was back in 1988. Like any kid of my age I was amazed by computer games. I started to learn programming with the hope that one day I will create my own game.

That happened after many years, in 1999 and here's the result. I wrote these two little games just to learn some Java language. Now looking back at this old code of mine, it's nice to see that I wasn't that bad after all :-)

Source code:

Click on images below to play!

When IHTMLWindow2.document throws UnauthorizedAccessException

2008-02-02T03:46:00.000-08:00

This is basically a C# translation of one of my older articles "When IHTMLWindow2::get_document returns E_ACCESSDENIED". Some .Net people encountered difficulties to use it, so I decided to make their life easier.

The main problem is the confusion created by System.IServiceProvider .Net interface because it has the same name as the COM interface. Once this issue is passed the code translation is straightforward. Here's the interop code to declare the COM interface IServiceProvider.

// This is the COM IServiceProvider interface, not System.IServiceProvider .Net interface!
[ComImport(), ComVisible(true), Guid("6D5140C1-7436-11CE-8034-00AA006009FA"),
InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]
public interface IServiceProvider
{
   [return: MarshalAs(UnmanagedType.I4)][PreserveSig]
   int QueryService(ref Guid guidService, ref Guid riid, [MarshalAs(UnmanagedType.Interface)] out object ppvObject);
}

You find here full source code of the sample assembly.

This technique was successfully implemented and tested in Twebst web automation library.

Nunit and STAThread story

2008-01-09T16:10:00.000-08:00

I use NUnit unit-testing framework to test my pet project Twebst. Being a collection of COM objects, Twebst can be used within any environment that supports COM. That means it can be used from .Net languages like C#.

First I started by creating an assembly to be used from NUnit GUI. Some tests failed without an obvious reason. After some research I understood that the COM apartment must be STAThread. The threading model must be set before the thread is started but I don't have access to NUnit GUI main thread from my assembly.

One possible solution to this problem is to transform the assembly into an EXE application that uses the NUnit framework like this:

[STAThread]
public static void Main(string[] args)
{
NUnit.ConsoleRunner.Runner.Main(
    new string[] { System.AppDomain.CurrentDomain.BaseDirectory + "MyExe.exe", "/nothread" });
}

When /nothread command line flag is used the tests are executed by the main thread which already has the right COM apartment properly set.

Repairing Internet Explorer

2007-12-20T04:06:00.001-08:00

Creating Internet Explorer extensions is not always an easy task. Erroneous extensions can make your IE to crash, become unstable or to behave unpredictably.

Even worse, uninstalling extensions can leave IE in a broken state, some things stop working properly. This usually happens when IE components are unregistered and IE registry keys are altered and not properly restored on uninstall. Here's a short list of damages I personally encountered:

1). 'Open in New Window' Command Does Not Work in Internet Explorer

2). When you try to add a Search Provider in Internet Explorer 7, a script error occurs, and the provider is not added successfully.
3). When you open a new tab in Internet Explorer 7, does the You've opened a new tab message appear every time, and are you unable to turn off this option
4). Open a link in a new tab takes forever to load.

I'll come later with examples of how an extension can break IE and how to avoid it. For now let's concentrate on how we can fix it. Download RepairIE.zip file, extract files inside and run "fixie.cmd"

Points of interest:

Internet Explorer components are registered using regsvr32.exe tool.
For IE7 mshtml.dll can not be registered as explained above; instead mshtml.tlb is registered using regtlib.exe tool.
Registering shdocvw.dll using regsvr32.exe is not a good idea on IE7 because it damages [HKEY_CLASSES_ROOT\Typelib\{EAB22AC0-30C1-11CF-A7EB-0000C05BAE0B}\1.1\0\win32 registry key.
reg_ieframe.reg file fixes the broken key above.

Allow local scripted HTML files to run in IE7

2007-12-09T02:11:00.000-08:00

During automatically testing Twebst library I was very annoyed about IE7 refusing to properly open local HTML files that contains scripts. The following message is displayed:

"To help protect your security, Internet Explorer has restricted this web page from running scripts or ActiveX controls that could access your computer. Click here for options..."

It took me several minutes to find the hidden option that turns off this warning. First I looked for it in Security tab options but it was actually in Advanced tab. Here's how you find it:

1). Go to:
Tools > Internet Options > Advanced > Security
2). Check:
Allow active content to run in files on My computer
3). Restart IE

How to get a handle to current TabWindowClass tab in IE7

2007-11-14T08:43:00.000-08:00

IWebBrowser2::get_HWND method gets the handle of the Internet Explorer 7 main window. Sometimes the tab window handle is needed. Here's a sample code that I recently found in MSDN. It shows how to get the handle of the tab window starting from a IWebBrowser2 object.

#include <shlguid.h>

HWND GetTabWnd(CComQIPtr<IWebBrowser2> spBrowser)
{
     HWND hwndTab = NULL;
     CComQIPtr<IServiceProvider> spServiceProvider = spBrowser;

     if (spServiceProvider != NULL)
     {
          CComQIPtr<IOleWindow> spWindow;
          if (SUCCEEDED(spServiceProvider->QueryService(
                         SID_SShellBrowser,
                              IID_IOleWindow,
                              (void**)&spWindow)))
          {
             
               spWindow->GetWindow(&hwndTab));
          }
     }

     return hwndTab;
}

I think the code is supposed to work on top level IWebBrowser2 objects. You can read more about top browser objects in my previous article.

This technique was successfully implemented and tested in Twebst web automation library.

When IWebBrowser2::get_HWND returns E_FAIL

2007-11-12T06:14:00.000-08:00

IWebBrowser2::get_HWND "gets the handle of the Microsoft Internet Explorer main window". As any COM method get_HWND returns a HRESULT value. According to MSDN, the method "returns S_OK if successful, or an error value otherwise".

It was hard for me to imagine how this method could fail but I still got an E_FAIL return value. This happened because the IWebBrowser2 object was not the top level browser. A web page containing frames/iframes is represented by a hierarchy of IHTMLWindow objects. Each window has an associated IHTMLDocument2 object exposed by IHTMLWindow2::get_document. An IHTMLWindow can be also converted to a IWebBrowser2 object. Here's my solution to get the main window handle starting from a non-top level browser object (this is a common scenario when adding your custom menu item in the IE context menu).

// IHTMLWindow2 to IWebBrowser2
CComQIPtr<IWebBrowser2> IHTMLWindow2ToIWebBrowser2(CComQIPtr<IHTMLWindow2> spHTMLWindow)
{
     ATLASSERT(spHTMLWindow != NULL);

     // Query for a service provider.
     CComQIPtr<IWebBrowser2>     spBrowser;
     CComQIPtr<IServiceProvider> spServiceProvider = spHTMLWindow;

     if (spServiceProvider != NULL)
     {
          // Ask the service provider for a IWebBrowser2 object.
          spServiceProvider->QueryService(IID_IWebBrowserApp, IID_IWebBrowser2, (void**)&spBrowser);
     }

     return spBrowser;
}

// IWebBrowser2 to IHTMLWindow2
CComQIPtr<IHTMLWindow2> IWebBrowserToIHTMLWindow(CComQIPtr<IWebBrowser2> spBrowser)
{
     ATLASSERT(spBrowser != NULL);
     CComQIPtr<IHTMLWindow2> spWindow;

     // Get the document of the browser.
     CComQIPtr<IDispatch> spDisp;
     spBrowser->get_Document(&spDisp);

     // Get the window of the document.
     CComQIPtr<IHTMLDocument2> spDoc = spDisp;
     if (spDoc != NULL)
     {
          spDoc->get_parentWindow(&spWindow);
     }

     return spWindow;
}


CComQIPtr<IWebBrowser2> TopBrowser(CComQIPtr<IWebBrowser2> spBrowser)
{
     ATLASSERT(spBrowser != NULL);

     // Retrieve IHTMLWindow2 from browser.
     CComQIPtr<IHTMLWindow2> spHTMLWnd = IWebBrowserToIHTMLWindow(spBrowser);
     if (spHTMLWnd != NULL)
     {
          // Find top window.
          CComQIPtr<IHTMLWindow2> spTopWindow;
          HRESULT hResult = spHTMLWnd->get_top(&spTopWindow);

          if (SUCCEEDED(hResult) && (spTopWindow != NULL))
          {
               // Convert the browser object to window.

               return IHTMLWindow2ToIWebBrowser2(spTopWindow);
          }
     }

     return CComQIPtr<IWebBrowser2>();
}

This technique was successfully implemented and tested in My web automation library.

How to programmatically find if XP theme is active?

2007-11-05T04:25:00.000-08:00

Registry key:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\ThemeManager

Value:
ThemeActive is "1" if Windows XP, "0" if Windows Classic.

Open TextRange selection as URL

2007-10-28T12:20:00.000-07:00

On many forums, people post URLs as plain text and this forces me to:

select the text URL
ctrl+C to copy it in the clipboard
ctrl+T to open a new tab
paste the URL in the address bar of the newly created tab
press "Enter" key to start navigation

I found this whole process very annoying and it seems I have to go through this many times a day. Here's my attempt to automate it by creating an extension for "Internet Explorer" similar to Linkification extension (for "Fire Fox" browser). I called my extension: "Make Link".

How it works:

right click on text URL
choose "MakeLink: use clicked text as link" menu item (the URL is open in a new tab automatically)

or you can

fully select the URL text or only a part of it
right click on selected text
choose "MakeLink: use clicked text as link" menu item (text link is open in a new tab automatically)

DOWNLOADS:

Points of interests:

HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\MenuExt registry key to add a item in the IE context menu
gain access to window object passed in external.menuArguments
document.selection property to get the selected TextRange
TextRange methods: moveStart, moveEnd, moveToPoint, select
The extension is entirely written in JScript (no BHO, no COM).

Known issues:

on some web pages with horizontal scroll simply right click a text URL won't work. You need to select a part of the URL and then right click it.

How to properly catch RBN_CHEVRONPUSHED notification?

2007-10-17T08:35:00.000-07:00

This is actually a thread I started on MSDN forum but unfortunately it remained unanswered:

Virtually any IE toolbar needs a chevron to happily live along with other toolbars in the same re-bar. So does my toolbar. To implement chevron functionality in IE toolbars I need to handle RBN_CHEVRONPUSHED. According to MSDN, when the chevron button is pushed, the notification is sent by the rebar in the form of WM_NOTIFY message to its parent. Here is the windows hierarchy in IE7:

WorkerW <- ReBarWindow32 <- ToolbarWindow32

where the last toolbar window is my toolbar. So I need to catch notifications from ReBarWindow32 that are sent to WorkerW window. To do that the first idea that came to my mind was to subclass the WorkerW window. I don’t like this idea because:

I subclass a window that does not belong to me, it was created by IE.
I don’t know what is the best time to subclass it: on IObjectWithSite.SetSite or on IDockingWindow.ShowDW ? (Those functions are implemented by my toolbar component)
I don’t know what is the best time to un-subclass it.
I don’t know when other toolbar might subclass/un-subclass the same window (I actually got a conflict with other toolbar resulting in IE stack overflow crash because of the order of subclassing/unsubclassing).

My second approach uses RB_SETPARENT to modify the parent of ReBarWindow32 window to be one of my windows. I process the RBN_CHEVRONPUSHED notification for my chevron button and send the other notifications to the original parent window (that is WorkerW). I change the parent on toolbar initialization/un-init (IObjectWithSite.SetSite). It seems a safer approach but I’m still worried about other toolbars using the same technique and the possibility of conflicts.

Take care of standard IE "Links" toolbar that also sends WM_COMMAND, WM_DRAWITEM and WM_MEASUREITEM messages to WorkerW window (and now you'll get those messages too). On IE6, "Go" button also do the same.

So the question remains: what is the best way to catch RBN_CHEVRONPUSHED notification when creating an IE toolbar extension?

When IHTMLWindow2::get_document returns E_ACCESSDENIED

2007-10-10T07:01:00.000-07:00

Internet Explorer extensions usually needs to access HTML elements. When extensions are initialized they get a IWebBrowser2 pointer representing the browser. Starting with this pointer one can get any HTML element in the web page but to do that we need to browse a hierarchy of frames first. The simplest web pages only have one frame and one document. Web pages containing <frame> or <iframe> have a hierarchy of frames, each frame having its own document.

Here are the objects involved and the corresponding interfaces:

browser      - IWebBrowser2
frame/iframe - IHTMLWindow2
document     - IHTMLDocument2
element      - IHTMLElement

The list bellow shows what method to call to get one object from another:

browser      -> document        IWebBrowser2::get_Document
document     -> frame           IHTMLDocument2::get_parentWindow
frame        -> document        IHTMLWindow2::get_document
frame        -> parent frame    IHTMLWindow2::get_parent
frame        -> children frames IHTMLWindow2::get_frames

A normal call chain to get a HTML element is:

browser -> document -> frame -> child frame -> ... -> child frame -> document -> element

This will work almost all the time. The problems arise when different frames contain documents loaded from different internet domains. In this case IHTMLWindow2::get_document returns E_ACCESSDENIED when trying to get the document from the frame object. I think this happens to prevent cross frame scripting atacks.

Here is HtmlWindowToHtmlDocument function I wrote to be used instead IHTMLWindow2::get_document to bypass the restriction:


// Converts a IHTMLWindow2 object to a IHTMLDocument2. Returns NULL in case of failure.
// It takes into account accessing the DOM across frames loaded from different domains.
CComQIPtr<IHTMLDocument2> HtmlWindowToHtmlDocument(CComQIPtr<IHTMLWindow2> spWindow)
{
     ATLASSERT(spWindow != NULL);

     CComQIPtr<IHTMLDocument2> spDocument;
     HRESULT hRes = spWindow->get_document(&spDocument);
    
     if ((S_OK == hRes) && (spDocument != NULL))
     {
          // The html document was properly retrieved.
          return spDocument;
     }

     // hRes could be E_ACCESSDENIED that means a security restriction that
     // prevents scripting across frames that loads documents from different internet domains.
     CComQIPtr<IWebBrowser2>  spBrws = HtmlWindowToHtmlWebBrowser(spWindow);
     if (spBrws == NULL)
     {
          return CComQIPtr<IHTMLDocument2>();
     }

     // Get the document object from the IWebBrowser2 object.
     CComQIPtr<IDispatch> spDisp;
     hRes = spBrws->get_Document(&spDisp);
     spDocument = spDisp;

     return spDocument;
}


// Converts a IHTMLWindow2 object to a IWebBrowser2. Returns NULL in case of failure.
CComQIPtr<IWebBrowser2> HtmlWindowToHtmlWebBrowser(CComQIPtr<IHTMLWindow2> spWindow)
{
     ATLASSERT(spWindow != NULL);

     CComQIPtr<IServiceProvider>  spServiceProvider = spWindow;
     if (spServiceProvider == NULL)
     {
          return CComQIPtr<IWebBrowser2>();
     }

     CComQIPtr<IWebBrowser2> spWebBrws;
     HRESULT hRes = spServiceProvider->QueryService(IID_IWebBrowserApp, IID_IWebBrowser2, (void**)&spWebBrws);
     if (hRes != S_OK)
     {
          return CComQIPtr<IWebBrowser2>();
     }

     return spWebBrws;
}

Here is the C# version of the code: "When IHTMLWindow2.document throws UnauthorizedAccessException".

This technique was successfully implemented and tested in My web automation library.

Quick and dirty

2007-10-05T06:44:00.000-07:00

One of the most interesting comments I ever found in source code is
//QD:

I am familiar with the well known //TODO: style of comment used by Visual Studio, but QD was new to me. As I found later, QD stands for "quick and dirty" and it is basically a low quality piece of code that will cause you a lot of trouble in the future. A quick search throughout the sources revealed plenty of QD comments. It seemed someone made a way of living out of this.
But sometimes programmers gotta do what programmers gotta do and even if I don't like it, I have to be quick and dirty.

Few days ago I was working on Windows mobile and I needed to decode a string that was HTTP posted to a web server. This is best accomplished by HttpUtility.UrlDecode method. Out of luck! HTTPUtility class is not available on .Net Compact Framework.

Here is where .Net Reflector came to help. I simply decompiled UrlDecode method in .Net Framework and copy-pasted the code to mobile project. Luckily, the generated code was short and it doesn't reference a lot of other classes. I was amazed by the quality of the decompiled code. Reflector did a great job!

Yes, it was quick, it was dirty but also it was very effective. As a bonus, the code is better than any code I can possibly produce in this very short time. After all, it was tested by Microsoft QA team!

From IAccessible to IHTMLElement and back

2007-10-03T06:20:00.000-07:00

When developing Internet Explorer plug-ins you might want to take advantage of the dual nature of the browser objects. Internet Explorer exposes the DOM (document object model) using IHTMLElement interface as the building block of the hierarchy. It also offers accessible objects through Active Accessibility IAccessible interface.



// From IAccessible to IHTMLElement.
CComQIPtr<IHTMLElement> AccessibleToHTMLElement(IAccessible* pAccessible)
{
     ATLASSERT(pAccessible != NULL);

     // Query for IServiceProvider interface.
     CComQIPtr<IServiceProvider> spServProvider = pAccessible;
     if (spServProvider != NULL)
     {
         // Ask the service for a IHTMLElement object.
         CComQIPtr<IHTMLElement> spHtmlElement;
         HRESULT hRes = spServProvider->QueryService(IID_IHTMLElement, IID_IHTMLElement,
                                                      (void**)&spHtmlElement);

         return spHtmlElement;
     }

     return CComQIPtr<IHTMLElement>();
}

// From IHTMLElement to IAccessible.
CComQIPtr<IAccessible> HTMLElementToAccessible(IHTMLElement* pHtmlElement)
{
     ATLASSERT(pHtmlElement != NULL);

     // Query for IServiceProvider interface.
     CComQIPtr<IServiceProvider> spServProvider = pHtmlElement;
     if (spServProvider != NULL)
     {
         // Ask the service for a IAccessible object.
         CComQIPtr<IAccessible> spAccessible;
         HRESULT hRes = spServProvider->QueryService(IID_IAccessible, IID_IAccessible,
                                                      (void**)&spAccessible);

         return spAccessible;
     }

     return CComQIPtr<IAccessible>();
}

Not all IHTMLElement objects support Active Accessibility. Here is the list of the HTML elements that are also accessible elements:
A, AREA, BUTTON, INPUT type=BUTTON, INPUT type=RESET, INPUT type=SUBMIT, FRAME, IMG, INPUT type=checkbox, INPUT type=image, INPUT type=password, INPUT type=radio, MARQUEE, OBJECT, APPLET, EMBED, SELECT, TABLE, TD, TH, TEXTAREA, INPUT type=TEXT.

This technique was successfully implemented and tested in My web automation library.

COM number speller

2007-07-17T12:08:00.000-07:00

Some time ago, I was asked to provide a way to print numbers as text in Romanian language. Interestingly, I couldn't find an easy way to do this in Excel (Office 2003 as far as I remember). Searching the internet, I found other people having the same problem. Microsoft provides a solution for English language, but not for Romanian.

I decided to create a reusable piece of code that will be easily used in as many environments as possible. I chose COM and ATL to be the solution to my problem. This is also a good programming exercise for my rusty COM / C++ skills and I plan to use it as a template for all my future COM objects.

Points of interest:

error info support by implementing IErrorInfo interface.
Help in CHM format and context identifiers specified in MIDL source file.
BSTR manipulation using CComBSTR class provided by ATL.
IDispatch support, so the component can be used from scripting environments.
Spelling implementation itself and support for multiple languages (only Romanian and English for now).

To install the COM object just simply run Install.bat
Here is the Excel macro that makes use of NumberSpeller COM object:


Function Spell(n As Currency) As String
'Create the speller object.
Dim s As NumberSpellerLib.speller
Set s = New NumberSpellerLib.speller

Dim o As Object
Set o = s
o.Language = "ro"
Spell = o.Translate(n)
End Function

Downloads:

full source code, compiled DLL and CHM help: NumberSpeller.zip
C++ sample code: NumberSpellerSample1.zip
Excel sample file: NumberSpellerSample2.zip
JS sample file: NumberSpellerSample3.zip

Blogs and C++ syntax highlighting

2007-05-27T09:26:00.000-07:00

This wasn't supposed to be my first post. I wanted to start my blogging existence writing about something else. It proved that being a blogger is not as easy as it should be, especially when trying to post some C++ code. The online editor allows to directly modify the HTML code but this is not the simplest or most enjoyable thing to do. Because I wanted C++ syntax highlighting so hard I decided to create a little tool to automate this task as much as possible and to allow me to focus on the real subject.

The result is cpptohtml.exe that takes a .cpp file as a command line argument and generates a .html file. If no file name is provided as argument, a standard "file open" dialog allows the user to specify the source .cpp file. The tool also accepts /C as command line arguments. When used the source code is taken from clipboard and the HTML result is put also in clipboard.

I found creating a windows shortcut to cpptohtml.exe very useful. This way I can launch the tool by pressing a shortcut key combination (CTR+ALT+T).
I'm gonna use this tool like this:

open preferred IDE and copy the C++ source code into the clipboard.
launch the tool using the shortcut key combination. The HTML result is in the clipboard.
paste the formated HTML in the online blog editor.

Download:
You can find here the full source code and the executable. I used Visual C++ and LEX. The archive also contains a .js script to automate the shortcut creation. I am pretty happy with the final result and how C++, LEX and JScript combines to automate a boring repetitive task.

Beautiful the technology is!