Import Wizard

First Feedback

I tried to order my requirements by priority in regard to the goal that the functionality can be used as soon as possible by non-techy people who need just ONE large topic set.

1. The functionality must not require installing or patching anything. If necessary, a pre-configured copy of the downloadable zip installation archive should be provided, perhaps named "2.0b8 express".
2. Next priority is clarity that is easily achievable by a little text in the property panel, regarding the Document Topic requirements (which are hopefully omittable as soon as possible, see File Chooser below).
   1. instead of "connect a Document type...": "Create a Document type (by rightclicking the backdrop canvas) and use the "Choose" button to select a CSV file that you put in your (my user files)\deepamehta-files\documents folder. Then draw a connection from the Import Wizard icon to the Document icon. Rightclick the connector line > Retype > Assignment.
   2. Add "When done, rightclick the ImportWizard icon again and select "Load preview". (As long as the "wizard" steps are distributed among the right pane and the left pane, a hint is necessary when to switch.)
3. Column names: The expected option "First row contains column names" should be easy to implement as a dropdown option similar to "separated by", right? If so, a third option should be available and should, in fact, be the default: "Autosuggest" (First column is Topic name, second is Topic Description). The current behaviour should be option 3 "Advanced" and would need much more consideration before recommending it to normal users. (In particular, the mix of meta topics like the little grey unnamed one, and the column name Properties unuequal to Name or Description, is too confusing. And if the Name property can be assigned from custom field names(?), this would need detailed description).
4. File Chooser: Why can't the File chooser be used with a property as in the File topic type? What is, anyway, the relationship of importer instances, importer configuration (template), file, and file import later rerun? Does a specific importer instance have to be tied to a certain file for the entire lifecycle ? How would I get rid of the "import results" association when I want to add other, new topics? I think, if so much complications are due to the intended rollback, this should be rethought and perhaps postponed until after a general undo feature. I think it is normally sufficient to repeat only the preview, not the import itself. Ideally, all the confusion of "creating" one importer could be avoided by reusing the same one with ever new properties, if the file chooser was a property like the separator values.
5. Row Order: It is great that you can reveal all imported topics at once (even though this feature is to well hidden). But once the number of items rows larger, their arbitrary initial locations are no longer useful. I would prefer an orderly arrangement (perhaps columns of approximately screen height, adding new columns horizontally as needed).

In the current behavior, Cziksentmihail's "flow" commandment of keeping the user in control, is severely violated when the user sees the item on the right, knows where s/he wants to put it on the left, and still cannot drag it or otherwise control the location with direct manipulation but clicks as sees the location as a surprise.

Core Functionality: CSV

There is a basic functionality that shares similarities with all potential advanced import features, and should therefore be considered and implemented first: Import of a simply structured, two-column list. The two columns would contain the core constituents of very generic topics:

• a smaller text string - this would become the name of the topic, shown in the visual context on the backdrop canvas on the left,
• and a longer text string - this would become the verbal description of the topic, shown in the property panel on the right.

For such simple data structures, there is a very basic format that is used by a multitude of applications as a 'lingua franca' or exchange language/ format: CSV = Character separated Values.

Most prominently, database applications, plain text editors, and spreadsehet applications mediating between these two. CSV is often erroneoulsly referred to as comma separated values, but the most flexible separator catering for all of these apps ist the Tab.

Today's users are used to comfortable interfaces for getting their data from one of these applications to the other.

• In database apps such as MS Access, a multi-step wizard analyses the first few records and provides useful suggestions about how the respective columns should be mapped to the fields of the new database table being populated;
• in spreadsheet applications such as MS Excel, a very similar wizard is launched when the user invokes the "data" > "text in columns" function for a particular column of cells.

Common to both apps is the affordance that the user does not need to imagine and guess how their data will be tackled, rather, s/he actually sees it.

So, a core functionality would be a wizard that

• navigates the user to their CSV file (similarly as the existing file browse dialog for selecting the topicmap zip file),
• collects basic information about the input file structure (copycat Access or Excel),
• suggests / asks which column should go to the visual overview pane and which one to the verbal detail pane.

The resulting functionality can be roughly derived from the cumbersome manual process that I have tested on different occasions, at last in a scenario described under "Motivation" below:

• take a CSV input file such as this
• generating the topicmap XML file is described here in the old wiki (skip section 1 and replace the awk script in section 2 by this one and invoke it with "-F^t")
• follow the rest of the procedure, as in section 3 and 4.

The last steps are good example why the import wizard could give reason to a minor overhaul of some completely incomprehensible peculiarities, such as the requirement of the types definitions and the icons folder, see below under Accompanying Usability Measures.

Motivation

To get a sense of how the target audience will love the feature, it is probably the best to work with the resulting affordances as a hands-on experience. Of course, the preferred mixture of spatially visual, verbal, and other modalities is diverse with the different users and hence, not everybody will love the new affordances. However, it would be really useful if the developers themselves could gain at least a glimpse of an impression about what wonderful thinking support becomes possible with this usage scenario.

A few days ago, I found yet another great usage example, and this time I really think this will be impressive for many different temperaments:

As you might know, each year the Edge world question center asks one hundred and fifty of the greatest minds among scientists, writers, and thinkers, a very hard question. (This time it was "What will change everything".) The 150 answers are compiled in linear html pages, obviously arranged just by incoming date rather than by any topical coherence. I doubt that anyone has fun reading them just sequentially, so you end up skimming and arbitrarily picking a few of them.

What about having them all 150 on a large topic map? Here is such a map, and you can arrange and group and connect them at will. Isn't this a dazzling array of great ideas, by such smart people? If this still doesn't strike a chord with you, I can provide you with many other examples that, at least, intrigued me a lot.

Of course, the example also reveals some weaknesses of the current usability. In the case of these essay titles, the most salient shortcoming is obviously the inability to wrap a line, so the longer titles overlap and partly obscure the other ones.

On the other hand, such examples give also reason to far-reaching theoretical arguments in favor of DeepaMehta. Just take the keyword "connect" from above. Connecting concepts (or people, or resources, as per the metaphor of neural connections) has been the subject of a much-discussed Massive Open Online Couse about Connectivism that I had the pleasure to participate during the last fall semester. And if someone would try to argue that ordinary mindmapping or concept mapping applications would suffice to do this, as well, we can provide plenty of good arguments why DeepaMehta does this job by far better, in a novel, unique way.

Another occasion where we could jump in is the upcoming Conference about Improving Conferences (see Siemens' tdoay's announcent of a Ning space open to everybody). I think, conference session topics could equally well be visualized, and a previous sample showed the sessions of the SemTech conference.

• Just ran across another potential useful application: A kind of Card Sorting for survey responses. Here, a lot of free-text responses are listed in long linear text. If they were on a topicmap, they could be arranged and connected. The format is very simple: Just text paragraphs starting with "Response "(number). I would like to copy and paste the entire list into a buffer. A simple logic would discover the simple structure and would suggest that "Response xx" could be used as topic name, while the reminder would end up in the description pane. With a few clicks on "Next >" buttons, the new item would populate a new topicmap, and I could start shuffling them around, just like traditional paper cards with card sorting but with the affordances of digital technology, in particular to glue rubber bands that won't tear away when the topics are moved. Additionaly, I could annotate the salient headwords of the long continuous texts, particularly by double-clicking and making them bold.
  • Cherry of the cake ("Sahnehäubchen"): The boldified word group could magically jump into the topic name field, like Delicious tags jump into the tags field when posting a favorite.

In many cases, the initial topic name could be just a number which could be assigned by the import wizard, so that the format of the input from clipbard would eventually require no special structure at all, except paragraphs. For instance, if you want to digest a long article like I did last week with this one (21 paragraphs with no other structure) you would only need to paste it into DeepaMehta and start thinking about it with your head and your mouse... x28de 21-Jan-2009 10:40 GMT

Extension Opportunities

Common Formats beyond CSV

Two types of format come to mind:

• those that are particularly useful for the intended audience of people who use DeepaMehta very much focussed on content rather than technical geekiness: e. g., literature processing applications such as Endnote, Bibtex, Citavi? or desktop folder overviews (as alternative to hierarchical Personal Brain) or overviews of microcontent such as blog post?
• those formats that are, already, in a reanonbly structured shape, e. g. other map formats like Cmap? or others Semantic Web stuff such as RDF?

Help Prepare the CSV

Even more desirable but probably rather difficult: Copy any RTF structures of nested bullets, or tables from web pages, and paste them as topic names and descriptions. dapper.org does some web page scraping like this. For further study.

Accompanying Usability Measures

Occasions for polishing have been mention above, e. g. the possibility to wrap or truncate a topic name on the map. Others will be found along the way.

• I, for one, would fancy an out-of-the box topic for major generic topics, comparable to the cities on the geographic map (preferably just as prominently colored).
• And another generic association type which is stronger than the unobtrusive gray (in fact, this should be the default, and the gray generic association should be deliberately placed where the association ist particularly vague).

x28de 12-Jan-2009 23:42 GMT

First Import Concept

This text is the result of our dev-meeting from 02.02.09. We tried to meet the in here mentioned needs of a first importer. We also discussed some client side issues on creating a more guiding dialog but didn't came to a solution which would be in grasp for us.

1) The Aim is Data Import

While reading in a textfile with tab seperated values (tsv) DeepaMehta creates automatically new topics. For importing there is no need for a matching already imported topics, there are always new topics created.

Before the actual import process the user can get an overview about how many topics would be created at all and if the fields of the "Target Type" are displaying the content fields correctly. This will be on the basis of some exemplary topics and all their respective content fields.

Optional: an import process can be reverted, also later.

2) Conception

For each import process there will be topics of one DeepaMehta type ("TargetType") created and the column names match the property names. Mapping onto already existing TopicTypes from column names onto property names is possible through adjusting the column names in the document which is to be imported.

To get the overview about the upcoming import process the user is able to generate a report in form of a table out of which it becomes apparent which topics would be created with which contents.

Optional: It should be possible to trigger a command which deletes all topics created within one import process, also later. Since there is no general undo there is need for a straight forward solution to keep your corporate memory clean. The first envisioned solution would be a delete command which enables to delete all items within a search ton. The ton with all imported topics cold be easily created via triggering "What's related" command on the Importer Topic.

The import functionality firstly will be available for administrators of a DeepaMehta instance.

3) Usage

1) Customize eventually the column names within the text file which is to be imported.

2) Import the document into DeepaMehta (usual functionality)

3) Create a new import topic of type "Textfile Importer" through "Create" - "Administration" - "Textfile Importer"

4) Associate the importer with the document and the "Goaltype" through two associations of type "Assignment" from the direction of the import topic.

5) Generate Report through triggering the "Create Report" command from the context menu of the importer. The Report will be displayed in the Property Panel (as a Property of the importer)

6) Start the import process through "Import Textfile" (via the context menu of the importer) and DeepaMehta creates the topics and associate (Typ: "Association", Grey) them with the importer topic.

7) Optional: Revert the import process through revealing all imported topics (via "What's related" from the import topic). The resulting search ton can be deleted (via the context menu of the container)

Malte 07-Feb-2009 15:11 GMT