Blog de Pierrick Brihaye

Ici : http://pierrick-brihaye.nom.fr/

I’ve found this one in my family album. I can’t remember where it was taken (not in Chauny, for sure) nor when, although probably in 1986 or so.

From left to right : myself (standing), Michel Babilotte, unknown, † Karim Kanane, Frank Wasserman (?), Laurent Béard (?), Erwan Brihaye.

1. The Netiquette patrol is watching
2. Discussing with Wolfgang of the current issues we have with our Collections.
3. Back to the hotel.
4. Everybody (myself, Leif-Jöran, Chris, one more Chris, Dannes and Wolfgang) is here. Errr… where’s Adam ? With Abi, for sure
5. On our way to the conference with Sabine and Leif-Jöran.
6. Showing to Leif-Jöran how the smoke gets out… sometimes Excellent shot from Dannes
7. Lunch
8. Lunch (not the same one and thus NOT the result of the preceding one ) Another excellent shot from Dannes !
9. Presenting my work on GML indexes
10. Adding “a few” items to the TODO list…

I have plugged a spatial index for GML geometries in eXist. Thanks to the new modularized index architecture, the developement was pretty easy and exciting.

Get there to learn more.

Comme on me le demande beaucoup ces derniers temps, je mets à disposition cet article non publié, relatif à la genèse du dossier électronique à l’Inventaire de Bretagne. De l’eau a coulé sous les ponts depuis sa rédaction en 2002, mais il y a sans doute encore des choses à y puiser.

At last : http://www4.culture.fr/patrimoines/patrimoine_architectural_et_mobilier/sribzh/main.xsp

sdxall:1

… is the magic formula to get everything

Really a nice event (http://www.xmlprague.cz/index.html) and a magnificent city with a strong continental climate. See the photos there, there and there especially :

• http://dilemma.ow.nl/prague/slides/IMG_0214.html (deep meditation)
• http://www.xmlprague.cz/album06/img030.jpeg.html (who’s the one without any glass ?)
• http://dilemma.ow.nl/prague/slides/IMG_0263.html (discussing eXist doctrinals with Chris Wallace and Wolfgang Meyer)
• http://dilemma.ow.nl/prague/slides/IMG_0267.html (probably listening to totally uninteresting stuff from DizZzZ. In the meanwhile, Chris tries to keep polite )
• http://picasaweb.google.com/WolfgangMM/XMLPrague/photo#4946519314958123026 (identifying the monuments around the Old Town square with Sabine)
• http://picasaweb.google.com/WolfgangMM/XMLPrague/photo#4946521697020346386 (DizZzZ, Chris, myself, Leif-Jöran, Sabine)
  
• http://dilemma.ow.nl/prague/slides/IMG_0325.html (the dark side of eXist’s IRC channel)

Téléchargeable ici.

Thanks to the perfectly reproductible bug#1400246, I now have a better view of what happens of, rather, what does not happen.

After long investigations, I’ve come to the conclusion that the way node contexts are handled is deficient ans should be refactored.

But… what is a node context ? Well, my answer will be a developer’s one and will thus be very far from the canonical definition given by the XPath 2.0 and XQuery specs :

A node context tells the current process why a node is here to be processed

If this definition is not clear enough (although I find it very nice), we will consider this XPath example :
/a/b/c[d/e/f]

Nodes a have document roots as context because they are children of such nodes. From eXist’s point of view, a context which consists of a document root is seen as being “empty” context (although we could imagine breaking the implicit intra-document contextualization granularity, but this is another story). Beware : an empty context is not a null one which used to make the impossible statement that children nodes can be the children of “nothing” true until recently (we now hav an exception as per the specs) .

Nodes b have a nodes as context nodes, c have b nodes as context, which themselves have a nodes as context and so forth…

Keeping track of node contexts is usually not necessary because we use a top/down traversing. When we process b, we don’t care any more if it was provided by a nor c by b.

However… when entering a predicate we must “mark” the current position before continuing the path evaluation. From then, we do need to keep track of the context nodes in order to apply the necessary filters when the marked position is restored.

To continue with our example, d is given c as context, e is given d as context and f is given e as context. The evaluation of the predicate, will consist in evaluating the existence of the f element child of e, child of d… We all know that eXist does this well and even pretty well ;-).

Now, when we restore the position, we want to check if the c element matches the positively evaluated predicate’s context, which is :
e\d\c (the backward slash is completely intented )

Since the predicate “reverse” context matches the current element, we can thus select this c element. This is how context evaluation is implemented. For most of the users, it works nicely.

However, if we use an XQuery like this :

let $t :=
<test>
<a><s>A</s> 1 </a>
<a><s>Z</s> 2 </a>
<a><s>B</s> 3 </a>
</test>
return $t//a[preceding-sibling::*[2]]


eXist will return the wrong result (<s>Z</s> 2 </a>, <a><s>B</s> 3 </a>) where we expect <a><s>A <a><s>Z</s> 2 </a>

Why ?

Because this is how the contexts nodes are “pushed” (the stack analogy is obvious) at the preceding-sibling step :

<a><s>A</s> 1 </a> has no preceding sibling. Nothing is done.
<a><s>Z</s> 2 </a> has <s>A</s> 1 </a> as preceding sibling. <s>A</s> 1 </a> is thus given <a><s>Z</s> 2 </a> as context.
<a><s>B</s> 3 </a> has (<s>A</s> 1 </a>, <a><s>Z</s> 2 </a>) as preceding-siblings. <s>A</s> 1 </a> and <a><s>Z</s> 2 </a> are thus given <a><s>B</s> 3 </a> as context.

Let us sum-up from a contextual point of view :
<a><s>A</s> 1 </a> has (<a><s>Z</s> 2 </a>\<a><s>B</s> 3 </a>) as context.
<a><s>Z</s> 2 </a> has (<a><s>B</s> 3 </a>) as context.
<a><s>B</s> 3 </a> has no context (it has never been matched).

From the preceding siblings, only <a><s>A</s> 1 </a> will be retained as a valid second preceding-sibling. This is how eXist evaluates the predicate and this evaluation is correct. A positional predicate is not a path expression step and thus does not modify the context of the retained nodes.

Now, what happens when eXists tries creates an the intersection of //a and the context of the predicate ? Well, it returns… (<s>Z</s> 2 </a>, <a><s>B</s> 3 </a>). Quod erat demonstrandum !

The explanation is relative to the very strange manner eXist has to compute this “intersection”. In fact, it matches all the nodes in the context of the nodes that are returned by the predicate evaluation.

In other terms, even if it pushes the context nodes as he would with a stack, it flattens it as (<s>Z</s> 2 </a>, <a><s>B</s> 3 </a>) at the end rather than considering it as a hierarchical path like <s>Z</s> 2 </a>\<a><s>B</s> 3 </a>, that would anyway be inaccurate since preceding-sibling is not a hierarchical axis.

I’ve thought about using the “pushed” nodes as if they were really stacked, using the last items in the stack (that means the first ones in eXist’s linked list) as a quick provisionnal fix. Unfortunately, this fix would only work for steps that return just one node (parent, self always do, other axis… sometimes do). My first task is therefore to clean the code so that it becomes able to add several context nodes to a context item.

I hope it will not be too long before :

• every evaluation step in a predicate creates a new context for the involved nodes.
• each of these contexts stores the context nodes of the nodes currently processed by the step. Ideally, the container should implement NodeSet, even though it should be one with reduced capabilities in order to reduce the overhead.

In the future, we may also consider a deferred, probaly slow, context computation because storing millions of context nodes to attached to nodes that will be filtered out is probably not worth the value in some cases. That is another story though.

Back to Golgoth 14’s bug which can now be reduced to :


declare variable $data as node() {
<data>
<folder id = "a">
<organisation>
<organisationid idRef="1"/>
<datefrom nil="true"/>
<dateto nil="true"/>
</organisation>
<organisation>
<organisationId idRef="2"/>
<dateFrom nil="true"/>
<dateTo>2005-10-01</dateTo>
</organisation>
</folder>
</data>
};

return
<r id=”{$folder/@id}”/>


As such it returns a result whereas is shouldn’t.

But replacing $folder/organisation[./organisationId/@idRef eq "2"] by $folder/organisation[organisationId/@idRef eq "2"] returns no result !

So, we definitely have a problem in the self axis…

Just as a coincidence, Timo Boehme made the same constatations just a few minutes later (http://article.gmane.org/gmane.text.xml.exist/7664) ! … and provided nice examples taken from the Shakespeare corpus.

From there, is was quite easy to discover what was going wrong. You will never believe where the bug was !

The issue is not closed however. We still have some context problems in some cases, maybe of the same nature than those in the attribute axis.

Still investigating.