Discussion:
[Xmldatadumps-l] change to output file numbering of big wikis
Ariel Glenn WMF
2018-05-31 11:36:51 UTC
Permalink
TL;DR:
Scripts that reply on xml files numbered 1 through 4 should be updated to
check for 1 through 6.

Explanation:

A number of wikis have stubs and page content files generated 4 parts at a
time, with the appropriate number added to the filename. I'm going to be
increasing that thi month to 6.

The reason for the increase is that near the end of the run there are
usually just a few big wikis taking their time at completing. If they run
with 6 processes at once, they'll finish up a bit sooner.

If you have scripts that rely on the number 4, just increase it to 6 and
you're done.

This will go into effect for the June 1 run and all runs afterwards.

Thanks!
Federico Leva (Nemo)
2018-05-31 12:21:27 UTC
Permalink
Post by Ariel Glenn WMF
The reason for the increase is that near the end of the run there are
usually just a few big wikis taking their time at completing. If they
run with 6 processes at once, they'll finish up a bit sooner.
Thanks for putting more CPUs at work for us. :)

Federico

Loading...