Version of collatedFileOperation with multiple read/write ranks. More...
Public Member Functions | |
| TypeName ("hostCollated") | |
| Runtime type information. More... | |
| hostCollatedFileOperation (const bool verbose) | |
| Construct null. More... | |
| virtual | ~hostCollatedFileOperation () |
| Destructor. More... | |
 Public Member Functions inherited from collatedFileOperation | |
| TypeName ("collated") | |
| Runtime type information. More... | |
| collatedFileOperation (const bool verbose) | |
| Construct null. More... | |
| collatedFileOperation (const label comm, const labelList &ioRanks, const word &typeName, const bool verbose) | |
| Construct from user communicator. More... | |
| virtual | ~collatedFileOperation () |
| Destructor. More... | |
| virtual fileName | objectPath (const IOobject &io, const word &typeName) const |
| Generate disk file name for object. Opposite of filePath. More... | |
| virtual bool | writeObject (const regIOobject &, IOstream::streamFormat format=IOstream::ASCII, IOstream::versionNumber version=IOstream::currentVersion, IOstream::compressionType compression=IOstream::UNCOMPRESSED, const bool write=true) const |
| Writes a regIOobject (so header, contents and divider). More... | |
| virtual void | flush () const |
| Forcibly wait until all output done. Flush any cached data. More... | |
| virtual word | processorsDir (const IOobject &) const |
| Actual name of processors dir. More... | |
| virtual word | processorsDir (const fileName &) const |
| Actual name of processors dir. More... | |
| virtual void | setNProcs (const label nProcs) |
| Set number of processor directories/results. Only used in. More... | |
| Public Member Functions inherited from masterUncollatedFileOperation | |
| TypeName ("masterUncollated") | |
| Runtime type information. More... | |
| masterUncollatedFileOperation (const bool verbose) | |
| Construct null. More... | |
| masterUncollatedFileOperation (const label comm, const bool verbose) | |
| Construct from communicator. More... | |
| virtual | ~masterUncollatedFileOperation () |
| Destructor. More... | |
| virtual bool | mkDir (const fileName &, mode_t=0777) const |
| Make directory. More... | |
| virtual bool | chMod (const fileName &, const mode_t) const |
| Set the file mode. More... | |
| virtual mode_t | mode (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Return the file mode. More... | |
| virtual fileType | type (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Return the file type: directory, file or link. More... | |
| virtual bool | exists (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Does the name exist (as directory or file) in the file system? More... | |
| virtual bool | isDir (const fileName &, const bool followLink=true) const |
| Does the name exist as a directory in the file system? More... | |
| virtual bool | isFile (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Does the name exist as a file in the file system? More... | |
| virtual off_t | fileSize (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Return size of file. More... | |
| virtual time_t | lastModified (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Return time of last file modification. More... | |
| virtual double | highResLastModified (const fileName &, const bool checkVariants=true, const bool followLink=true) const |
| Return time of last file modification. More... | |
| virtual fileNameList | readDir (const fileName &, const fileType=fileType::file, const bool filterVariants=true, const bool followLink=true) const |
| Read a directory and return the entries as a string list. More... | |
| virtual bool | cp (const fileName &src, const fileName &dst, const bool followLink=true) const |
| Copy, recursively if necessary, the source to the destination. More... | |
| virtual bool | ln (const fileName &src, const fileName &dst) const |
| Create a softlink. dst should not exist. Returns true if. More... | |
| virtual bool | mv (const fileName &src, const fileName &dst, const bool followLink=false) const |
| Rename src to dst. More... | |
| virtual bool | mvBak (const fileName &, const std::string &ext="bak") const |
| Rename to a corresponding backup file. More... | |
| virtual bool | rm (const fileName &) const |
| Remove a file, returning true if successful otherwise false. More... | |
| virtual bool | rmDir (const fileName &) const |
| Remove a directory and its contents. More... | |
| virtual fileName | filePath (const bool checkGlobal, const IOobject &, const word &typeName) const |
| Search for an object. checkGlobal : also check undecomposed case. More... | |
| virtual fileName | dirPath (const bool checkGlobal, const IOobject &) const |
| Search for a directory. checkGlobal : also check undecomposed. More... | |
| virtual fileNameList | readObjects (const objectRegistry &db, const fileName &instance, const fileName &local, word &newInstance) const |
| Search directory for objects. Used in IOobjectList. More... | |
| virtual bool | readHeader (IOobject &, const fileName &, const word &typeName) const |
| Read object header from supplied file. More... | |
| virtual autoPtr< ISstream > | readStream (regIOobject &, const fileName &, const word &typeName, const bool read=true) const |
| Reads header for regIOobject and returns an ISstream. More... | |
| virtual bool | read (regIOobject &, const bool masterOnly, const IOstream::streamFormat defaultFormat, const word &typeName) const |
| Top-level read. More... | |
| virtual autoPtr< ISstream > | NewIFstream (const fileName &filePath, IOstream::streamFormat format=IOstream::ASCII, IOstream::versionNumber version=IOstream::currentVersion) const |
| Generate an ISstream that reads a file. More... | |
| virtual autoPtr< Ostream > | NewOFstream (const fileName &filePath, IOstream::streamFormat format=IOstream::ASCII, IOstream::versionNumber version=IOstream::currentVersion, IOstream::compressionType compression=IOstream::UNCOMPRESSED, const bool write=true) const |
| Generate an Ostream that writes a file. More... | |
| virtual label | addWatch (const fileName &) const |
| Add watching of a file. Returns handle. More... | |
| virtual bool | removeWatch (const label) const |
| Remove watch on a file (using handle) More... | |
| virtual label | findWatch (const labelList &watchIndices, const fileName &) const |
| Find index (or -1) of file in list of handles. More... | |
| virtual void | addWatches (regIOobject &, const fileNameList &) const |
| Helper: add watches for list of regIOobjects. More... | |
| virtual fileName | getFile (const label) const |
| Get name of file being watched (using handle) More... | |
| virtual void | updateStates (const bool masterOnly, const bool syncPar) const |
| Update state of all files. More... | |
| virtual fileMonitor::fileState | getState (const label) const |
| Get current state of file (using handle) More... | |
| virtual void | setUnmodified (const label) const |
| Set current state of file (using handle) to unmodified. More... | |
| virtual instantList | findTimes (const fileName &, const word &) const |
| Get sorted list of times. More... | |
| virtual IOobject | findInstance (const IOobject &io, const scalar startValue, const word &stopInstance) const |
| Find instance where IOobject is. Fails if cannot be found. More... | |
| virtual void | setTime (const Time &) const |
| Callback for time change. More... | |
| const HashPtrTable< instantList > & | times () const |
| Return cached times. More... | |
| Public Member Functions inherited from fileOperation | |
| TypeName ("fileOperation") | |
| Runtime type information. More... | |
| fileOperation (const label comm) | |
| Construct null. More... | |
| declareRunTimeSelectionTable (autoPtr, fileOperation, word,(const bool verbose),(verbose)) | |
| virtual | ~fileOperation () |
| Destructor. More... | |
| virtual fileName | filePath (const fileName &) const |
| Search for a file or directory. Use IOobject version in. More... | |
| virtual label | nProcs (const fileName &dir, const fileName &local="") const |
| Get number of processor directories/results. Used for e.g. More... | |
| fileName | processorsCasePath (const IOobject &, const word &procDir) const |
| Generate path (like io.path) from root+casename with any. More... | |
| fileName | processorsPath (const IOobject &, const word &instance, const word &procDir) const |
| Generate path (like io.path) with provided instance and any. More... | |
| fileName | processorsPath (const fileName &, const word &procDir) const |
| Operating on fileName: replace processorXXX with procDir. More... | |
Additional Inherited Members | |
| Public Types inherited from fileOperation | |
| enum | pathType { NOTFOUND, ABSOLUTE, OBJECT, WRITEOBJECT, PROCUNCOLLATED, PROCBASEOBJECT, PROCOBJECT, PARENTOBJECT, FINDINSTANCE, PROCUNCOLLATEDINSTANCE, PROCBASEINSTANCE, PROCINSTANCE } |
| Enumeration for the location of an IOobject. More... | |
| typedef Tuple2< fileName, Tuple2< pathType, label > > | dirIndex |
| Description of processor directory naming: More... | |
| typedef List< dirIndex > | dirIndexList |
| Static Public Member Functions inherited from masterUncollatedFileOperation | |
| static bool | uniformFile (const fileNameList &) |
| Same file? More... | |
| Static Public Member Functions inherited from fileOperation | |
| static autoPtr< fileOperation > | New (const word &type, const bool verbose) |
| Select type. More... | |
| static label | splitProcessorPath (const fileName &, fileName &path, fileName &procDir, fileName &local, label &groupStart, label &groupSize, label &nProcs) |
| Split fileName into part before 'processor' and part after. More... | |
| static label | detectProcessorPath (const fileName &) |
| Detect processor number from '/aa/bb/processorDDD/cc'. More... | |
| Static Public Attributes inherited from collatedFileOperation | |
| static float | maxThreadFileBufferSize |
| Max size of thread buffer size. This is the overall size of. More... | |
| Static Public Attributes inherited from masterUncollatedFileOperation | |
| static float | maxMasterFileBufferSize |
| Max size of parallel communications. Switches from non-blocking. More... | |
| Static Public Attributes inherited from fileOperation | |
| static const NamedEnum< pathType, 12 > | pathTypeNames_ |
| static word | processorsBaseDir = "processors" |
| Return the processors directory name (usually "processors") More... | |
| static word | defaultFileHandler |
| Default fileHandler. More... | |
| static autoPtr< fileOperation > | fileHandlerPtr_ |
| Static fileOperation. More... | |
| Protected Member Functions inherited from collatedFileOperation | |
| bool | isMasterRank (const label proci) const |
| Is proci master of communicator (in parallel) or master of. More... | |
| bool | appendObject (const regIOobject &io, const fileName &filePath, IOstream::streamFormat fmt, IOstream::versionNumber ver, IOstream::compressionType cmp) const |
| Append to processors/ file. More... | |
| Protected Member Functions inherited from masterUncollatedFileOperation | |
| template<class Type > | |
| Type | scatterList (const UList< Type > &, const int, const label comm) const |
| template<class Type , class fileOp > | |
| Type | masterOp (const fileName &, const fileOp &fop, const int tag, const label comm) const |
| template<class Type , class fileOp > | |
| Type | masterOp (const fileName &, const fileName &, const fileOp &fop, const int tag, const label comm) const |
| virtual fileName | filePathInfo (const bool checkGlobal, const bool isFile, const IOobject &, pathType &searchType, word &processorsDir, word &instance) const |
| Search (locally!) for object; return info on how it was found. More... | |
| fileName | localObjectPath (const IOobject &, const pathType &searchType, const word &processorsDir, const word &instancePath) const |
| Construct filePath. More... | |
| bool | exists (const dirIndexList &, IOobject &io) const |
| Helper: check IO for local existence. Like filePathInfo but. More... | |
| Protected Member Functions inherited from fileOperation | |
| fileMonitor & | monitor () const |
| void | cacheProcessorsPath (const fileName &fName) const |
| Detect presence of processorsDDD. More... | |
| tmpNrc< dirIndexList > | lookupProcessorsPath (const fileName &) const |
| Lookup name of processorsDDD using cache. Return empty fileName. More... | |
| bool | exists (IOobject &io) const |
| Does ioobject exist. Is either a directory (empty name()) or. More... | |
| Static Protected Member Functions inherited from collatedFileOperation | |
| static labelList | ioRanks () |
| Static Protected Member Functions inherited from masterUncollatedFileOperation | |
| static labelList | subRanks (const label n) |
| Get the list of processors that are part of this communicator. More... | |
| static word | findInstancePath (const instantList &timeDirs, const instant &t) |
| Equivalent of Time::findInstance. More... | |
| static void | readAndSend (const fileName &fName, const labelUList &procs, PstreamBuffers &pBufs) |
| Detect file (possibly compressed), read file contents and send. More... | |
| static autoPtr< ISstream > | read (IOobject &io, const label comm, const bool uniform, const fileNameList &filePaths, const boolList &read) |
| Read files on comms master. More... | |
| Static Protected Member Functions inherited from fileOperation | |
| static instantList | sortTimes (const fileNameList &, const word &) |
| Sort directory entries according to time value. More... | |
| static void | mergeTimes (const instantList &extraTimes, const word &constantName, instantList ×) |
| Merge two times. More... | |
| static bool | isFileOrDir (const bool isFile, const fileName &) |
| Helper: check for file (isFile) or directory (!isFile) More... | |
| Protected Attributes inherited from collatedFileOperation | |
| const label | myComm_ |
| Any communicator allocated by me. More... | |
| OFstreamCollator | writer_ |
| Threaded writer. More... | |
| label | nProcs_ |
| Number of processors (overall) More... | |
| const labelList | ioRanks_ |
| Ranks of IO handlers. More... | |
| Protected Attributes inherited from masterUncollatedFileOperation | |
| const label | myComm_ |
| Any communicator allocated by me. More... | |
| HashPtrTable< instantList > | times_ |
| Cached times for a given directory. More... | |
| Protected Attributes inherited from fileOperation | |
| const label | comm_ |
| Communicator to use. More... | |
| HashTable< dirIndexList > | procsDirs_ |
| Detected processors directories. More... | |
| autoPtr< fileMonitor > | monitorPtr_ |
| file-change monitor for all registered files More... | |
Version of collatedFileOperation with multiple read/write ranks.
In parallel it will assume ranks are sorted according to hostname and the lowest rank per hostname will be the IO rank. The output directories will get a unique name processors<N>_<low>-<high> where N is the overall number of processors and low and high is the range of ranks contained in the files. Each of these subsets uses its own communicator.
Instead of using the hostnames the IO ranks can be assigned using the FOAM_IORANKS environment variable (also when running non-parallel), e.g. when decomposing into 4:
FOAM_IORANKS='(0 2)' decomposePar -fileHandler hostCollated
will generate
processors4_0-1/ containing data for processors 0 to 1 processors4_2-3/ containing data for processors 2 to 3
Definition at line 72 of file hostCollatedFileOperation.H.
| hostCollatedFileOperation | ( | const bool | verbose | ) |
Construct null.
Definition at line 129 of file hostCollatedFileOperation.C.
References Foam::endl(), forAll, Pstream::gatherList(), Foam::hostName(), Foam::Info, UPstream::master(), UPstream::myProcNo(), Foam::name(), UPstream::nProcs(), and Foam::pid().
|
virtual |
Destructor.
Definition at line 169 of file hostCollatedFileOperation.C.
References UPstream::freeCommunicator().
| TypeName | ( | "hostCollated" | ) |
Runtime type information.
1.8.13 
