Form Panel

Settings

Using this panel you can have Blue Crab submit a simple form, via the HTTP "POST" method. This is useful if you need to first log into a web site before accessing its pages, or you want to crawl the response page of a form submission.

When you open a grabber window with a configuration that has a form submission there are two modes of operation possible:

• If the grabber has an associated URL then Blue Crab first submits the form and then starts crawling the associated URL. The purpose of this is to enable the ability to log into a website before crawling pages it contains. For example, this can ensure Blue Crab obtains any necessary cookies associated with the login process.

• If the grabber does not have an associated URL then Blue Crab first submits the form and then starts crawling the response page of the form submission. This enables you to crawl a page which is only accessible by form submission.

An HTML form consists of a <FORM> tag followed by several input elements (text fields, popup menus, radio buttons, etc.). Every form also has an asociated "action" which is a URL that specifies where to send the fom data. This panel enables you to specify values for all of these items as described below. In particular the table at the bottom of the panel consists of name-value pairs which correspond to the form input elements.

You have the option of manually entering all information, or using the various buttons to automate part of the process. Either way you should always view the source HTML of the form in your web browser to make sure you have properly configured the form submission. In fact, certain form elements require that you do so. For example, if a form contains a popup menu, then although Blue Crab may accurately extract the name of this element and add it to the table you need to determine the allowed values. You can do that by viewing the HTML source for the popup menu.

Note that Blue Crab does not currently support the "GET" method (so forms like Google won't work.)

Settings

• Enter the URL of the form.

  Enter the URL of the form here. Click the "Load and Parse" button to have Blue Crab extract the forms action and input items for you.

• Current Web Page:

  Click to copy the URL of the front window of your web browser (Safari, OmniWeb, Camino or Internet Explorer.) The form will also be loaded and parsed.

• Load and Parse:

  Click to load and parse the form. This will add its input items to the table below and extract the forms action URL. If a page has multiple forms then only the last form is currently extracted.

• Clear:

  Erase all entries on this panel.

• Form Action:

  Enter the URL of the forms action here. Blue Crab will start its crawl using this URL by submitting the values in the table below.

• Form Method:

  Select an HTTP method. (Only POST is currently supported.)

• Test Submit:

  Click to submit the form input item data to the action URL and save the response to a disk file.

• New:

  Create a new input item in the table.

• Delete:

  Delete an input item in the table.

• Edit:

  Edit an input item in the table.