Data storage, Visualization and Data Analysis

Simulations running on modern HPC machines can produce large amounts of scientific data, that sometimes need to be post-processed with data analytics and visualization tools. Starting from mid 2014 CINECA provides a facility to better face these kind of applications. This system, called PICO, consists of a smaller cluster of standard processors, with a higher amount of central memory and accelerators for enhance the rendering process. PICO also has faster direct access to the shared data repository to allow fast reading of data from simulations.

PICO is a BigData system specifically tailored for data-analysis and visualization. A large and fast data repository (nearly 4 PB, called $DRES) is connected to PICO and accessible in a shared way with all the HPC systems. It is intended for sharing data within the project team and across computational platforms. $DRES is a multi-level storage via LTFS.  A new Tape library is connected to PICO, equipped with 12 PB (expandible to 16PB) of space for uncompressed data. It is available only though the $DRES filesystem with the LTFS technology. A specific entry is defined for each username ($TAPE)  in a shared way across all HPC systems, for private archiving of data on magnetic support for long time archive.

These shared areas complement the usual local storage areas, HOME, SCRATCH and WORK. 

• Home: it is intended for source codes, executebles, small data files.
  Local on each HPC system, each user has an entry, pointed by $HOME env variable
• Scratch: it is intended for the output of the batch jobs.
  Local on each HPC system, each user has an entry, pointed by $CINECA_SCRATCH env variable
• Work: it is intended for the output of the batch jobs, for secure sharing within the project team.
  Local on each HPC system (at present only FERMI), each project has an entry , pointed by $WORK env variable
• Dres: It is intended as a medium/long term repository and as a shared area within the project tema and across HPC platforms.
  Shared area (PICO compute nodes and login-nodes of HPC systems), project area, pointed by $DRES env variable
• Tape: it is intended as a personal long term archive area.
  Shared area, each user has an entry, pointed by $TAPE env variable (via LTFS)