Generate a persist file per code-hash

Redbook:

In previous versions of the platform a persist generated a single output file with a name that matched the string supplied within the persist(). It was rebuilt if a query was submitted which was based on different code, or the input files have changed.

In 4.2 this has changed so that the output filename is derived by appending a unique identifier derived from the ECL code onto the end of the filename. This means if there are two users using old/new versions of the same persist they can both rebuild and co-exist independently. There are a couple of implications that may cause backward compatibility problems.

• Disk usage may go up because there are more copies of the persist. This can be alleviated by using the new persist options (), or manually deleting out of date persists.

• Very occasionally ECL users have directly declared the files as ECL DATASETs and read them directly. (This isn't recommended!) If this is being employed then the persist definitions should have the ,SINGLE attribute added onto them to ensure the unmodified filename is used.

The following options have been added as options to PERSIST:

SINGLE - use the old semantics of a single persist file which matches the name of the persist. This is useful for unusual situations where someone is directly reading the persist file.

MULTIPLE - use a different persist name for different ECL definitions of the persist. (The name is a combination of the name supplied by the user suffixed with a 32bit value derived from the ECL.)

MULTIPLE - keep up to n different versions of the persist file. Without the argument the system default is used.

#option ('multiplePersistInstances', true|false)

can be used to control the default behaviour for the workunit. False is the previous semantics, true is the new default semantics.

I can add the last bit of information into the redbook but it might help to also have details of the symptoms the user may experience if they haven't done what you recommend and are looking to find information.

Also, are you able to say which releases this might affect (and I am assuming I can say fixed in 4.2)

The plan is to do the following:

The code generator to add an option to mangle the persist name with the hash of the ecl code. The default is controlled by a debug option ('multiplePersistInstances'), and can be overridden by adding , SINGLE or ,MANY to the PERSIST operator.

When enabled, all the persist names and filenames will have a suffix added to them. Options are either "::<ecl-hash>" or "__<ecl-hash>". If :: is used the same persists will be generated into a separate file scope. That may cause problems, may have advantages. I suspect we will go with "__".

This does mean that any code that attempts to read a persist file directly either needs to append the ecl-hash to the end of the filename, or it needs to specify ,SINGLE on the persist definition. [ redbook]