Crthuky

Yesterday, I imported a large set of data into a Mathematica notebook and stored each imported list of numbers in a function. For example, I would map a lists like 10, 20, 30 to a function values as shown below

f[0] = 10, 20 30;
f[1] = 40, 50, 60;

With the lists stored in the functions I generated the below chat by writing

averageComparisonChart = 
 BarChart[fpAverages, fpiAverages, 
 ChartLabels -> "FP Quicksort", "FP Insertion Quicksort", 
 Range[0, 160, 10], 
 AxesLabel -> HoldForm["Vector size"], 
 HoldForm["Execution time (ms)"], 
 PlotLabel -> HoldForm["Quicksort vs. Insertion sort"], 
 LabelStyle -> GrayLevel[0]]

which output

Before going to bed, I saved my notebook and shut down my computer. Today, all my functions have been reset. For example inputing f[0] outputs f[0] rather than the previously assigned list 10, 20, 30.

Does anyone know what has caused this issue? How can loss of data be avoided in the future? Is there a better way to store lists than in functions? Is there a way to restore the values from yesterday?

Related Question

The accepted answer to this question provides a method for creating persistence of data between sessions.

If you wrap your definitions in Once then their results will be remembered across sessions:

f[0] = Once[Print["a"]; 10, 20, 30, "Local"]

Here the printing and the numbers 10, 20, 30 are used instead of a lengthy calculation that you only want to do once and whose result you want to remember in the next session.

On the first execution, the above code prints "a" and assigns the numbers 10, 20, 30 to f[0]. On subsequent executions (even after you've closed Mathematica and come back and are reevaluating the notebook), the execution of the first argument of Once does not take place any more, so there is no printing, and only the remembered result 10, 20, 30 is directly assigned to f[0]. This speeds up the reprocessing on subsequent executions dramatically if the list 10, 20, 30 is replaced with something hard to compute.

With Once you don't need to save/restore semi-manually as some comments suggest with Save, DumpSave, Get. Instead, persistent storage operates transparently to cache what has been calculated before.

Once has more options: you can specify in which cache the persistent storage should be (in the front end session, or locally so that even when you close and reopen Mathematica it's still there) and how long it should persist.

Another way to create persistent objects is with PersistentValue, which is a bit lower-level than Once but basically the same mechanism.

How to get rid of persistent objects

A certain wariness with persistent storage is in order. But note that these Once definitions are associated with a hash of the expression executed as the first argument to Once, and not with where you're storing the result (as in f[0] in my example). In this sense I consider Once a safe and unconfusing technique to use; I haven't run into trouble with undesired cross-contamination of unconditionally persistent objects. No persistent storage will be consulted unless you explicitly wrap an expression in Once.

Nonetheless in practice I keep the persistent storage pool as clean as possible. As the documentation states, you can inspect the storage pool with

PersistentObjects["Hashes/Once/*"]

PersistentObject["Hashes/Once/BlVsTGCUwUI", PersistentLocation[..., Type:Local]],
PersistentObject["Hashes/Once/FziAfp1s_y2", PersistentLocation[..., Type:Local]]

and clean it with

DeleteObject[%]

Like in all other systems I am familiar with, variable and function definitions exist in memory (RAM) only and do not persist across sessions.

If you want a definition to persist, you must save it explicitly. See Save and DumpSave.

However, what I recommend for cases like yours is not to store such data in DownValue definitions. Store them in a data structure that is easy to serialize, then save them to a file. So, instead of f[1]=a; f[2]=b; f[3]=c use a list a,b,c. If the indices are not contiguous, you can use a SparseArray or Association. You can save any data that is stored as a Mathematica expression into an MX file, which is the most practical and flexible format for short-term storage (not for archiving because of weak cross-version compatibility promises). For archiving or for exchange with other systems, consider JSON: any expression that consists of lists, associations, numbers and strings can be saved to JSON.