Public Member Functions | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | List of all members
collatedFileOperation Class Reference

Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory. More...

Inheritance diagram for collatedFileOperation:
Inheritance graph
[legend]
Collaboration diagram for collatedFileOperation:
Collaboration graph
[legend]

Public Member Functions

 TypeName ("collated")
 Runtime type information. More...
 
 collatedFileOperation (const bool verbose)
 Construct null. More...
 
 collatedFileOperation (const label comm, const labelList &ioRanks, const word &typeName, const bool verbose)
 Construct from user communicator. More...
 
virtual ~collatedFileOperation ()
 Destructor. More...
 
virtual fileName objectPath (const IOobject &io, const word &typeName) const
 Generate disk file name for object. Opposite of filePath. More...
 
virtual bool writeObject (const regIOobject &, IOstream::streamFormat format=IOstream::ASCII, IOstream::versionNumber version=IOstream::currentVersion, IOstream::compressionType compression=IOstream::UNCOMPRESSED, const bool valid=true) const
 Writes a regIOobject (so header, contents and divider). More...
 
virtual void flush () const
 Forcibly wait until all output done. Flush any cached data. More...
 
virtual word processorsDir (const IOobject &) const
 Actual name of processors dir. More...
 
virtual word processorsDir (const fileName &) const
 Actual name of processors dir. More...
 
virtual void setNProcs (const label nProcs)
 Set number of processor directories/results. Only used in. More...
 
- Public Member Functions inherited from masterUncollatedFileOperation
 TypeName ("masterUncollated")
 Runtime type information. More...
 
 masterUncollatedFileOperation (const bool verbose)
 Construct null. More...
 
 masterUncollatedFileOperation (const label comm, const bool verbose)
 Construct from communicator. More...
 
virtual ~masterUncollatedFileOperation ()
 Destructor. More...
 
virtual bool mkDir (const fileName &, mode_t=0777) const
 Make directory. More...
 
virtual bool chMod (const fileName &, const mode_t) const
 Set the file mode. More...
 
virtual mode_t mode (const fileName &, const bool followLink=true) const
 Return the file mode. More...
 
virtual fileName::Type type (const fileName &, const bool followLink=true) const
 Return the file type: DIRECTORY, FILE or LINK. More...
 
virtual bool exists (const fileName &, const bool checkGzip=true, const bool followLink=true) const
 Does the name exist (as DIRECTORY or FILE) in the file system? More...
 
virtual bool isDir (const fileName &, const bool followLink=true) const
 Does the name exist as a DIRECTORY in the file system? More...
 
virtual bool isFile (const fileName &, const bool checkGzip=true, const bool followLink=true) const
 Does the name exist as a FILE in the file system? More...
 
virtual off_t fileSize (const fileName &, const bool followLink=true) const
 Return size of file. More...
 
virtual time_t lastModified (const fileName &, const bool followLink=true) const
 Return time of last file modification. More...
 
virtual double highResLastModified (const fileName &, const bool followLink=true) const
 Return time of last file modification. More...
 
virtual fileNameList readDir (const fileName &, const fileName::Type=fileName::FILE, const bool filtergz=true, const bool followLink=true) const
 Read a directory and return the entries as a string list. More...
 
virtual bool cp (const fileName &src, const fileName &dst, const bool followLink=true) const
 Copy, recursively if necessary, the source to the destination. More...
 
virtual bool ln (const fileName &src, const fileName &dst) const
 Create a softlink. dst should not exist. Returns true if. More...
 
virtual bool mv (const fileName &src, const fileName &dst, const bool followLink=false) const
 Rename src to dst. More...
 
virtual bool mvBak (const fileName &, const std::string &ext="bak") const
 Rename to a corresponding backup file. More...
 
virtual bool rm (const fileName &) const
 Remove a file, returning true if successful otherwise false. More...
 
virtual bool rmDir (const fileName &) const
 Remove a directory and its contents. More...
 
virtual fileName filePath (const bool checkGlobal, const IOobject &, const word &typeName) const
 Search for an object. checkGlobal : also check undecomposed case. More...
 
virtual fileName dirPath (const bool checkGlobal, const IOobject &) const
 Search for a directory. checkGlobal : also check undecomposed. More...
 
virtual fileNameList readObjects (const objectRegistry &db, const fileName &instance, const fileName &local, word &newInstance) const
 Search directory for objects. Used in IOobjectList. More...
 
virtual bool readHeader (IOobject &, const fileName &, const word &typeName) const
 Read object header from supplied file. More...
 
virtual autoPtr< ISstreamreadStream (regIOobject &, const fileName &, const word &typeName, const bool valid=true) const
 Reads header for regIOobject and returns an ISstream. More...
 
virtual bool read (regIOobject &, const bool masterOnly, const IOstream::streamFormat format, const word &typeName) const
 Top-level read. More...
 
virtual autoPtr< ISstreamNewIFstream (const fileName &) const
 Generate an ISstream that reads a file. More...
 
virtual autoPtr< OstreamNewOFstream (const fileName &pathname, IOstream::streamFormat format=IOstream::ASCII, IOstream::versionNumber version=IOstream::currentVersion, IOstream::compressionType compression=IOstream::UNCOMPRESSED, const bool valid=true) const
 Generate an Ostream that writes a file. More...
 
virtual label addWatch (const fileName &) const
 Add watching of a file. Returns handle. More...
 
virtual bool removeWatch (const label) const
 Remove watch on a file (using handle) More...
 
virtual label findWatch (const labelList &watchIndices, const fileName &) const
 Find index (or -1) of file in list of handles. More...
 
virtual void addWatches (regIOobject &, const fileNameList &) const
 Helper: add watches for list of regIOobjects. More...
 
virtual fileName getFile (const label) const
 Get name of file being watched (using handle) More...
 
virtual void updateStates (const bool masterOnly, const bool syncPar) const
 Update state of all files. More...
 
virtual fileMonitor::fileState getState (const label) const
 Get current state of file (using handle) More...
 
virtual void setUnmodified (const label) const
 Set current state of file (using handle) to unmodified. More...
 
virtual instantList findTimes (const fileName &, const word &) const
 Get sorted list of times. More...
 
virtual IOobject findInstance (const IOobject &io, const scalar startValue, const word &stopInstance) const
 Find instance where IOobject is. Fails if cannot be found. More...
 
virtual void setTime (const Time &) const
 Callback for time change. More...
 
const HashPtrTable< instantList > & times () const
 Return cached times. More...
 
- Public Member Functions inherited from fileOperation
 TypeName ("fileOperation")
 Runtime type information. More...
 
 fileOperation (const label comm)
 Construct null. More...
 
 declareRunTimeSelectionTable (autoPtr, fileOperation, word,(const bool verbose),(verbose))
 
virtual ~fileOperation ()
 Destructor. More...
 
virtual fileName filePath (const fileName &) const
 Search for a file or directory. Use IOobject version in. More...
 
virtual label nProcs (const fileName &dir, const fileName &local="") const
 Get number of processor directories/results. Used for e.g. More...
 
fileName processorsCasePath (const IOobject &, const word &procDir) const
 Generate path (like io.path) from root+casename with any. More...
 
fileName processorsPath (const IOobject &, const word &instance, const word &procDir) const
 Generate path (like io.path) with provided instance and any. More...
 
fileName processorsPath (const fileName &, const word &procDir) const
 Operating on fileName: replace processorXXX with procDir. More...
 

Static Public Attributes

static float maxThreadFileBufferSize
 Max size of thread buffer size. This is the overall size of. More...
 
- Static Public Attributes inherited from masterUncollatedFileOperation
static float maxMasterFileBufferSize
 Max size of parallel communications. Switches from non-blocking. More...
 
- Static Public Attributes inherited from fileOperation
static const NamedEnum< pathType, 12 > pathTypeNames_
 
static word processorsBaseDir = "processors"
 Return the processors directory name (usually "processors") More...
 
static word defaultFileHandler
 Default fileHandler. More...
 
static autoPtr< fileOperationfileHandlerPtr_
 Static fileOperation. More...
 

Protected Member Functions

bool isMasterRank (const label proci) const
 Is proci master of communicator (in parallel) or master of. More...
 
bool appendObject (const regIOobject &io, const fileName &pathName, IOstream::streamFormat fmt, IOstream::versionNumber ver, IOstream::compressionType cmp) const
 Append to processors/ file. More...
 
- Protected Member Functions inherited from masterUncollatedFileOperation
template<class Type >
Type scatterList (const UList< Type > &, const int, const label comm) const
 
template<class Type , class fileOp >
Type masterOp (const fileName &, const fileOp &fop, const int tag, const label comm) const
 
template<class Type , class fileOp >
Type masterOp (const fileName &, const fileName &, const fileOp &fop, const int tag, const label comm) const
 
virtual fileName filePathInfo (const bool checkGlobal, const bool isFile, const IOobject &, pathType &searchType, word &processorsDir, word &instance) const
 Search (locally!) for object; return info on how it was found. More...
 
fileName localObjectPath (const IOobject &, const pathType &searchType, const word &processorsDir, const word &instancePath) const
 Construct filePath. More...
 
bool exists (const dirIndexList &, IOobject &io) const
 Helper: check IO for local existence. Like filePathInfo but. More...
 
- Protected Member Functions inherited from fileOperation
fileMonitormonitor () const
 
void cacheProcessorsPath (const fileName &fName) const
 Detect presence of processorsDDD. More...
 
tmpNrc< dirIndexListlookupProcessorsPath (const fileName &) const
 Lookup name of processorsDDD using cache. Return empty fileName. More...
 
bool exists (IOobject &io) const
 Does ioobject exist. Is either a directory (empty name()) or. More...
 

Static Protected Member Functions

static labelList ioRanks ()
 
- Static Protected Member Functions inherited from masterUncollatedFileOperation
static labelList subRanks (const label n)
 Get the list of processors that are part of this communicator. More...
 
static word findInstancePath (const instantList &timeDirs, const instant &t)
 Equivalent of Time::findInstance. More...
 
static void readAndSend (const fileName &filePath, const IOstream::compressionType cmp, const labelUList &procs, PstreamBuffers &pBufs)
 Read file contents and send to processors. More...
 
static void readAndSend (const fileName &fName, const labelUList &procs, PstreamBuffers &pBufs)
 Detect file (possibly compressed), read file contents and send. More...
 
static autoPtr< ISstreamread (IOobject &io, const label comm, const bool uniform, const fileNameList &filePaths, const boolList &procValid)
 Read files on comms master. More...
 
- Static Protected Member Functions inherited from fileOperation
static instantList sortTimes (const fileNameList &, const word &)
 Sort directory entries according to time value. More...
 
static void mergeTimes (const instantList &extraTimes, const word &constantName, instantList &times)
 Merge two times. More...
 
static bool isFileOrDir (const bool isFile, const fileName &)
 Helper: check for file (isFile) or directory (!isFile) More...
 

Protected Attributes

const label myComm_
 Any communicator allocated by me. More...
 
OFstreamCollator writer_
 Threaded writer. More...
 
label nProcs_
 Number of processors (overall) More...
 
const labelList ioRanks_
 Ranks of IO handlers. More...
 
- Protected Attributes inherited from masterUncollatedFileOperation
const label myComm_
 Any communicator allocated by me. More...
 
HashPtrTable< instantListtimes_
 Cached times for a given directory. More...
 
- Protected Attributes inherited from fileOperation
const label comm_
 Communicator to use. More...
 
HashTable< dirIndexListprocsDirs_
 Detected processors directories. More...
 
autoPtr< fileMonitormonitorPtr_
 file-change monitor for all registered files More...
 

Additional Inherited Members

- Public Types inherited from fileOperation
enum  pathType {
  NOTFOUND, ABSOLUTE, OBJECT, WRITEOBJECT,
  PROCUNCOLLATED, PROCBASEOBJECT, PROCOBJECT, PARENTOBJECT,
  FINDINSTANCE, PROCUNCOLLATEDINSTANCE, PROCBASEINSTANCE, PROCINSTANCE
}
 Enumeration for the location of an IOobject. More...
 
typedef Tuple2< fileName, Tuple2< pathType, label > > dirIndex
 Description of processor directory naming: More...
 
typedef List< dirIndexdirIndexList
 
- Static Public Member Functions inherited from masterUncollatedFileOperation
static bool uniformFile (const fileNameList &)
 Same file? More...
 
- Static Public Member Functions inherited from fileOperation
static autoPtr< fileOperationNew (const word &type, const bool verbose)
 Select type. More...
 
static label splitProcessorPath (const fileName &, fileName &path, fileName &procDir, fileName &local, label &groupStart, label &groupSize, label &nProcs)
 Split fileName into part before 'processor' and part after. More...
 
static label detectProcessorPath (const fileName &)
 Detect processor number from '/aa/bb/processorDDD/cc'. More...
 

Detailed Description

Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory.

Uses threading if maxThreadFileBufferSize > 0.

See also
masterUncollatedFileOperation
Source files

Definition at line 58 of file collatedFileOperation.H.

Constructor & Destructor Documentation

◆ collatedFileOperation() [1/2]

collatedFileOperation ( const bool  verbose)

◆ collatedFileOperation() [2/2]

collatedFileOperation ( const label  comm,
const labelList ioRanks,
const word typeName,
const bool  verbose 
)

Construct from user communicator.

Definition at line 338 of file collatedFileOperation.C.

References Foam::endl(), IOobject::fileModificationChecking, Foam::Info, IOobject::inotifyMaster, Foam::nl, IOobject::timeStampMaster, and WarningInFunction.

Here is the call graph for this function:

◆ ~collatedFileOperation()

~collatedFileOperation ( )
virtual

Destructor.

Definition at line 403 of file collatedFileOperation.C.

References UPstream::freeCommunicator(), and collatedFileOperation::objectPath().

Here is the call graph for this function:

Member Function Documentation

◆ ioRanks()

Foam::labelList ioRanks ( )
staticprotected

Definition at line 75 of file collatedFileOperation.C.

References Foam::getEnv(), collatedFileOperation::ioRanks(), and collatedFileOperation::isMasterRank().

Referenced by collatedFileOperation::ioRanks().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ isMasterRank()

bool isMasterRank ( const label  proci) const
protected

Is proci master of communicator (in parallel) or master of.

the io ranks (non-parallel)

Definition at line 91 of file collatedFileOperation.C.

References collatedFileOperation::appendObject(), Foam::findIndex(), UPstream::master(), and UPstream::parRun().

Referenced by collatedFileOperation::ioRanks().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ appendObject()

bool appendObject ( const regIOobject io,
const fileName pathName,
IOstream::streamFormat  fmt,
IOstream::versionNumber  ver,
IOstream::compressionType  cmp 
) const
protected

◆ TypeName()

TypeName ( "collated"  )

Runtime type information.

◆ objectPath()

Foam::fileName objectPath ( const IOobject io,
const word typeName 
) const
virtual

Generate disk file name for object. Opposite of filePath.

Reimplemented from fileOperation.

Definition at line 415 of file collatedFileOperation.C.

References IOobject::instance(), masterUncollatedFileOperation::localObjectPath(), word::null, fileOperation::OBJECT, TimePaths::processorCase(), fileOperation::PROCOBJECT, IOobject::time(), and collatedFileOperation::writeObject().

Referenced by collatedFileOperation::~collatedFileOperation().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ writeObject()

bool writeObject ( const regIOobject io,
IOstream::streamFormat  format = IOstream::ASCII,
IOstream::versionNumber  version = IOstream::currentVersion,
IOstream::compressionType  compression = IOstream::UNCOMPRESSED,
const bool  valid = true 
) const
virtual

◆ flush()

void flush ( ) const
virtual

Forcibly wait until all output done. Flush any cached data.

Reimplemented from masterUncollatedFileOperation.

Definition at line 611 of file collatedFileOperation.C.

References Foam::endl(), masterUncollatedFileOperation::flush(), Foam::Pout, and collatedFileOperation::processorsDir().

Here is the call graph for this function:

◆ processorsDir() [1/2]

Foam::word processorsDir ( const IOobject io) const
virtual

Actual name of processors dir.

Reimplemented from fileOperation.

Definition at line 689 of file collatedFileOperation.C.

References IOobject::objectPath().

Referenced by collatedFileOperation::flush(), and collatedFileOperation::processorsDir().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ processorsDir() [2/2]

Foam::word processorsDir ( const fileName fName) const
virtual

Actual name of processors dir.

Reimplemented from fileOperation.

Definition at line 625 of file collatedFileOperation.C.

References forAll, UList< T >::last(), Foam::name(), UPstream::nProcs(), UPstream::parRun(), collatedFileOperation::processorsDir(), UPstream::procID(), and List< T >::size().

Here is the call graph for this function:

◆ setNProcs()

void setNProcs ( const label  nProcs)
virtual

Set number of processor directories/results. Only used in.

decomposePar

Reimplemented from fileOperation.

Definition at line 697 of file collatedFileOperation.C.

References Foam::endl(), and Foam::Pout.

Here is the call graph for this function:

Member Data Documentation

◆ myComm_

const label myComm_
protected

Any communicator allocated by me.

Definition at line 67 of file collatedFileOperation.H.

◆ writer_

OFstreamCollator writer_
mutableprotected

Threaded writer.

Definition at line 70 of file collatedFileOperation.H.

◆ nProcs_

label nProcs_
protected

Number of processors (overall)

Definition at line 75 of file collatedFileOperation.H.

◆ ioRanks_

const labelList ioRanks_
protected

Ranks of IO handlers.

Definition at line 78 of file collatedFileOperation.H.

◆ maxThreadFileBufferSize

float maxThreadFileBufferSize
static

Max size of thread buffer size. This is the overall size of.

all files to be written. Starts blocking if not enough size. Read as float to enable easy specification of large sizes.

Definition at line 111 of file collatedFileOperation.H.

Referenced by collatedFileOperationInitialise::needsThreading().


The documentation for this class was generated from the following files: