Mixed data search results are correct, or A ball is not a lady

At some blogging website, we have found an example of the ‘error’ in recognizing pictures at Picollator. The author has submitted the ball image together with the text query ‘ball’. Then he reported that Picollator recognized the ball as the lady’s face. Look at this screen shot below:

No similar images found there. Just text search works.

If you pay attention on the screen shot, you can find that:

1. A tomato sign in the upper right corner of the input image of the ball is red. It means that the user request contains that image as the sample
2. The text ‘ball’ is presented in the query string

Image of the ball did not return any results, because the recognition algorithms distinguish faces good enough. I should say that the results can be found by image OR text query. In our case Picollator has found portraits of that lady because her name contains ‘Ball’, i.e., by text query.

This is like when you submit long textual query to the conventional search engine, and it returns results with just a few words as not all words are found. Here, Picollator did the same. It did not find similar images, but retrieved images by textual user input.

How can you know which webresource is found by which data type? Of course, we are working on the interface issue, as it is not easy to demonstrate results by mixed request. At this moment, you may use one of the following ways:

1. Move the mouse cursor over the resource (image) and look at the pop-up. It contains either one found image, or submitted image and found one together. If there is only one picture, then it is found by text, which is displayed below. In case of two images first one is found by similarity with the submitted sample.
2. Move the mouse over the input image. If you locate the mouse in the facial area, you can see all similar images in the list of results if some of them are found by visual similarity. Same functionality works if you move the mouse over the input image face in the previous case.
3. You may use tomato sign to switch on and off the image-based search.

For more info, read my previous posts:

http://recogmission.wordpress.com/2008/04/17/identified-faces-are-highlighted-now/

http://recogmission.wordpress.com/2008/07/07/text-web-search-in-picollator/

Sorry, I have noticed that I did not say anything about this very new image+text interface with tomato before… Please give me more time.

Text search obtains semantics

As I stated in my past posting, we implemented the universal search at www.picollator.com . Using text, images or all of those together you can find web pages and images. Now I am writing a note about that intriguing ability of Picollatorto find the data using text much better than others. As I wrote, Picollator now understands the correspondence between visual objects and text submitted and indexed. Let me publish a couple of examples.

1 I put a simple text request with the word ‘barbara‘, which is just a female name, nothing else. With Picollator, I obtained following results:

text search using the word barbara at picollator.com

Compare them to ones from famous conventional text search system:

text search using the word barbara at other web search system

Put a closer look. First one contains results ordered not just by the word ‘barbara’, but also by similarity between faces inside images. Last one contains just arbitrary or unknown ranking, while Picollator delivers most valuable images first. 

2 I put a simple text request with the word ‘sam‘, which is just a male name. With Picollator, I obtained following results:

text search using the word sam at picollator.com

Again, you can see the tendency: similar faces are displayed first and by groups in a contrast to disordered results of the corresponding search from other search engine:

text search using the word sam at other search engine

It is not just because we collect similar images and faces one by one – it is practically unrealistic in the scale of WWW. Picollator includes the technology which allows to compare visual objects with words, i.e., it uses semantics. It does it without any formal algebra approach, just because Picollator is able to visually compare pictures and objects, as well as collect and index text data.

Of course, it is the very first release of that tech, but it will grow, the database will become bigger, and you will see the web search engine with real A.I.. Maybe, it is alive already. Sometimes I feel like if it would be alive.

Text web search in Picollator

We have made a new release of www.picollator.com with the advanced text search. As you have seen, we made tags as key text to find a content. However, the new generation allows very simple text search using the input string as all web search engines do. Of course, it is not so powerful yet. However, it is a little bit more than just text-based search. First of all, you can search using text input (as you probably use to do in Google, Yahoo, MSN, Ask, etc.). Second, you can search using visual input as Picollator allowed initially. Third, you can search using image and text as the query at the same time, and it is the thing Picollator offers as the technology pioneer again. The ranking is performed across all resources found depending on image-based ratios and text-based ratios together. There is a very small thing about the text search I have to tell you about. This text search is not just a regular text search like ranked text index results. It depends on the visual similarity of some data processed, even if you use only text to search. Picollator is able to – please pay attention on what I am saying! – Picollator is able to understand the visual meaning of some words. It does not mean that it just count the words attached to the pictures or webpages. Its engine also uses some estimation of how text can fit visual similarity of known objects.

In my next posting I will publish some advises on using that new release.