Differences in `40008005` vs. `41000018` DIS Grids

[Radonirinaunimi]

Theory ID 41000018 uses the exact same settings as 40008005 and thus the two theories should be exactly the same up to numerical errors (~1e-7). And this is exactly the case for all the nFONLL sub-theories except for 03 (massive component of FFNS $n_f=4$) and 04 (FFN0 $n_f=4$).

Here is for instance the comparisons for the HERA NC (which for the record is the one that presents the largest differences): comparisons.txt.

Looking more closely into the contents of the grids, order-by-order, we can see that even the orders that overlap between the two present large differences: comparisons_ID03_per_order.txt.

We know structurally from various PRs (namely NNPDF/pineko#248 and #346) that new theories contain more orders but we do expect those extra-orders to be (numerically) zeros. However, this does not also seem to be the case.

On the other hand, the logs from the run (results_4100001803.log), which compares the results from the PineAPPL interpolations and the direct yadism calculations, suggests that the differences originate from yadism.

[scarlehoff]

I guess if the differences are at the level of grids, it's only from yadism that they can come (provided the input cards are the same).

The only possible saving grace, and I'm not sure whether you tested this already, is that due to nFONLL the extra pieces cancel out when you use the right PDF for the comparison (i.e., one with nf=3 or nf=4).
Just to be clear, I don't think this is going to be the case, but it's the last thing that can occur at the grid level. Anything else has to come from yadism I think?

But this is great that now we know exactly which pieces are wrong, a good old git bisect should tell us exactly what changed when :D

[Radonirinaunimi]

I will bisect commits tomorrow but as far as this issue is concerned a1ebe2f2 reproduces the old results. The bad news is that this is far far back in the history xd

[Radonirinaunimi]

Okay, so I am not sure if this is a good or a bad news but the situation is as follows.

After a bit of fighting with git bisect (unfortunately it was impossible to automate because each commit has its own python restrictions and requires different input cards) I managed to track down the commit responsible for the difference. It is e14f54f3 that was introduced in #262 to address the issue #260.

That PR essentially introduces as default intrinsic configurations for heavy quarks, while before it was left to the user to turn it on. This means that theory 40008005 was generated without IC.

[juanrojochacon]

@Radonirinaunimi many thanks for the investigation. This seems a very serious bug (so it is good that we found it).

I am not sure I understand what do you mean by "This means that theory 40008005 was generated without IC". Without IC where? You mean in the DIS grids?

From your message above it seems that " And this is exactly the case for all the nFONLL sub-theories except for 03 and 04, which are the massive matrix elements. So I guess you mean that in the massive matrix elements for DIS structure functions, there was no charm-initiated contributions, and that the charm PDF appeared only via the massless piece?

[juanrojochacon]

In any case I am very happy that we have identified the problem @Radonirinaunimi @scarlehoff @felixhekhorn at last my understanding is that the "NNPDF4.1 DIS theories" are correct (in that they account for the IC contribution in the FFNS and FFN0 grids) and that the bug was limited to "40008005", which if I look in the wiki is "only" used for the alphas paper https://www.wiki.ed.ac.uk/spaces/nnpdfwiki/pages/652698412/Recent+versions+of+fit+variants, where IC effects are not huge.

If this understanding is correct, I guess there is not much we can do but it would be very good to recompute 40008005 with the correct settings so that we have a debugged baseline for NNPDF4.1, else we will be missing different effects. And we also want to do the same at N3LO, where I guess this bug was also there for the NNPDF4.0 alphas paper and now has been corrected?

[juanrojochacon]

From the physics point of view, from the comparisons we have done so far, my understanding of the consequences of this bug can be seen from the attached plot (look at the red dotted line, which is a DIS only so it quantifies the impact of this bug):

• The main impact is for the gluon at x \sim 0.001, where a shift of around half a sigma is found. At large differences are tiny (which is good for our IC claims, and consistent with the bug since FFN is relevant for medium and low-x data)
• For the other PDF flavours, differences are well below PDF errors
• The effect of this bug is quite smaller than other theory changes in 4.1, in particular the updated theories for jet production.

So all in all a very unfortunate bug, but its pheno impact on the PDFs is moderate thank heavens.

We can discuss tomorrow how to proceed, probably we should recompute the 4.0 DIS tables to see explicitely that this difference goes away.

@Radonirinaunimi I guess that by the transitivity property this bug will also affect the 4.0alphas N3LO grids right?

PDFs-pulls-new41base.pdf

[juanrojochacon]

PDFs-pulls-new41base-DIS.pdf

For completeness, this is the summary of the effects of the bug for the DIS only fits.

@kamillaurent do we have the updated aN3LO fits that we agreed to run with the new N3LO FK-tables? I think it would be important to quantify what is the impact of the bug there, so see to which extent this could have influenced our alphas extraction.

@Radonirinaunimi @felixhekhorn @scarlehoff can you please confirm the fits which have been affected by this bug? Only the alphas paper or also some other studies? For example maybe the mtop paper is also affected?

[felixhekhorn]

First of all: thank you @Radonirinaunimi for your work!

And this is exactly the case for all the nFONLL sub-theories except for 03 (massive component of FFNS n f = 4 ) and 04 (FFN0 n f = 4 ).

I managed to track down the commit responsible for the difference. It is e14f54f3 that was introduced in #262 to address the issue #260.

wait a second, there is something wrong: as we can see from the numbers, indeed massive or asymptotic $n_f=4$ calculations are different. However, in $n_f=4$ there is - by definition - no IC, because the charm PDF exists as massless PDF. So while it is very good to know the commit, e14f54f, the physics reason is not yet clear to me. I wonder if the bug was to double count charm in $n_f=4$ and to add the IC coefficient function by mistake

[juanrojochacon]

what do you mean in 4FFN there is no charm? There is a charm initiated (massive) contribution right @felixhekhorn ?

[felixhekhorn]

maybe I'm just confused, but @Radonirinaunimi and @scarlehoff , I have the vague feeling of a deja-vu ... did we trip over this commit already at some point in the past?

[felixhekhorn]

what do you mean in 4FFN there is no charm? There is a charm initiated (massive) contribution right @felixhekhorn ?

nope, by $n_f=4$ = FFN4 we mean four light flavors: u+d+s+c; so charm contributes (/should contribute) only through the massless coefficient functions. Potential intrinsic PDFs in this scheme are only bottom and top - both of which we (should) never consider.