VAST DATA LTD.

A method for safe storage of a data portion, the method includes (a) receiving, at a storage system, the data portion; (b) selecting, based on at least a timing parameter, a redundancy scheme for safe storage of the data portion; wherein the redundancy scheme is selected out of (i) mirroring the data portion and (ii) applying a stripe based protection on the data portion, at a storage space of the storage system; and (c) safe storing the data portion by applying the selected redundancy scheme.

A method for managing metadata blocks of file system entities (FSEs), the method includes (i) determining, by a processing circuit of a storage system, a stress level that is non-linearly dependent upon a fullness of static FSE metadata blocks and a fullness of dynamic FSE metadata blocks, wherein the static FSE metadata blocks store FSE metadata items of a first type and the dynamic FSE metadata blocks store FSE metadata items of a second type and additional FSE metadata items of a first type; (ii) determining, based on the stress level and a fullness parameter, a probability of applying a conditional compression process on a new FSE metadata block; and (iii) applying the conditional compression process, based on the probability, on the new FSE metadata block.

A method for monitoring largest file system entities of a file system that is stored in a storage system, the method may include populating, during multiple population iterations, a cache of the storage system with largest file system entities metadata; wherein a current population iteration of the multiple population iterations may include pushing, to a heap, metadata related to a set of one or more file system entities that are children of a parent file system entity that was deemed a largest file system entity of a last population iteration that preceded the current population iteration; popping the heap to provide a current largest file system entity; and storing metadata related to the current largest file system entity in the cache.

A method for updating a partially ordered database, the method includes (a) obtaining, by a merge circuit, density values of multiple sorted table chunks of the partially ordered database, wherein a density value of each sorted table chunk is indicative of a relationship between a number of populated rows of the sorted table chunk and a width of the sorted table chunk, wherein the width of the sorted table chunk is determined based on (i) values of sort keys of the sorted table chunk, and (ii) a parameter of a distribution of values of sort keys of the partially ordered database; each sort key is associated with a row of a table of the partially ordered database and represents values of multiple cells of the row; (b) merging, by the merge circuit, a set of sorted table chunks to provide a new sorted table chunk that exhibits a density value that exceeds a density value of each sorted table chunk of the set; and (c) replacing, in a storage unit of a storage system, the set of sorted table chunks by the new sorted table chunk.

A method for cluster based searching for a value range stored in a storage system, the method may include receiving a request to find a certain value range within a set of information elements that are stored in a storage system; wherein the set of information elements comprises subsets of information elements associated with subset hash based filters; wherein different subsets of information elements are associated with different subset hash based filters; determining a certain cluster value of a certain cluster that comprises the certain value range; applying one or more hush functions on the certain cluster value to provide one or more hash results; and determining whether one or more members of the certain cluster are possibly in a subset of information elements, based on the one or more hash results and on a subset hash based filter of the subset of information elements; and when determining that the one or more members of the certain cluster are possibly in the subset then searching, within the subset, a matching information element that matches the certain value range.

A method for managing write buffers (WBs), that includes (i) receiving write requests for writing input information units (IIUs) to the storage system; (ii) estimating an occurrence of a future overflow of first WBs that are allocated, during steady state operation, to store IIUs before the IIUs are processed and migrated to a permanent storage; and (iii) responding to the estimating by (a) storing at least some IIUs at second WBs without temporarily storing the at least some of the IIUs in the first WBs; (b) reducing a rate of a migration of IIUs from the first WBs and the second WBs to the permanent storage; and (c) introducing intentional delays in handling an ingestion of one or more newly received write requests.

A method for time-aligning data sent from a first storage system to a second storage system, the method includes (i) receiving multiple data units and a synchronization message, by a controller of the second storage system (SSS) and while applying a synchronous replication scheme; the multiple data units comprise first snapshot (S 1 ) data units and second snapshot (S 2 ) data units; the synchronization message indicates that a controller of the FSS has changed an allocation of data units from the S 1 to the S 2 ; and (ii) associating SSS timestamps with the multiple data units, wherein the associating of the SSS timestamps includes (a) associating a SSS timestamp related to the S 1 for each S 1 data unit, regardless a timing relationship between a time of reception of the synchronization message and a time of reception of the S 1 data unit; and (b) delaying an association of a SSS timestamp related to the S 2 for each S 2 data unit, until receiving the synchronization message.

A method for locking metadata, the method includes (i) executing a first FSE command, by obtaining a lock of FSE metadata that is related to the FSE and is stored at a remote shared storage unit, and creating and accessing a local version of the FSE metadata; (ii) estimating that a reception of the first FSE command will be followed, within a defined time frame, by a reception of a second FSE command; (iii) maintaining the lock following a completion of the first FSE command; and (iv) executing the second FSE command, following the reception of the second FSE command, wherein the executing of the second FSE command includes accessing the local version of the FSE metadata.

A method for managing file system entity (FSE) open requests in a shared namespace. According to an embodiment, the method includes (a) receiving, by a controller of a first storage system (FSS), a second storage system (SSS) transfer request for transferring to the SSS a responsibility to manage SSS given FSE open requests, the SSS given FSE open requests are generated by SSS users; (b) determining, by the controller of the FSS, whether to grant the SSS transfer request, wherein the determining is based on whether there is a current granted given FSE open request. When it is determined to grant the SSS transfer request the method includes (c) transferring the responsibility to manage the SSS given FSE related open requests to the SSS; and (d) updating, by the controller of the FSS, a FSS transfer data structure to reflect the transferring of the responsibility.

A method for metadata blocks access, the method includes (i) accessing, following a reception of an access request, a metadata block related to a file system entity; the metadata block is associated with a transaction identifier that is indicative of a transaction that is a most recent transaction that modified the metadata block; (ii) traversing, starting from the metadata block, metadata blocks that are transaction related, using linkage metadata within the metadata blocks; (iii) determining, based on an outcome of the traversing, a status of the transaction; (iv) when it is determined that the transaction was successfully completed then completing the access request by using content that was written by the transaction to the metadata block; and (v) when it is determined that the transaction failed then ignoring content that was written by the transaction to the metadata block.