rice.persistence
Class PersistentStorage

java.lang.Object
  rice.persistence.PersistentStorage

All Implemented Interfaces:

Catalog, Storage

public class PersistentStorage
extends java.lang.Object
implements Storage

This class is an implementation of Storage which provides persistent storage to disk. This class also guarentees that the data will be consistent, even after a crash. This class also provides these services is a non-blocking fashion by launching a seperate thread which is tasked with actaully writing the data to disk. This class was initially designed to support only Ids whose toString() method returns a String of constant length. It has been exteneded to support variable-length toString()s, but we have the caveat that not toString() can be a substring of another toString() - this will cause undefined behavior. Additionally, the toString() method on the key Ids *CANNOT* have the period ('.') or exclamation point ('!') characters in them - these are used for internal purposes. The serialized objects are stored on-disk in a GZIPed XML format, which provides extensibility with reasonable storage and processing costs. Additionally, any metadata, if provided, is also stored in the on-disk file. The format of the file is [Key, Object, Version, Gzipped XML] [Metadata, Gzipped XML] [persistence magic number, long] [persistence version, long] [persistence revision, long] [metadata length, long] The persistence package is set up to automatically upgrade older versions of the on-disk format as new data is written under the key. Persistence also supports the metadata interface specified in the Catalog interface. All metadata is guaranteeded to be stored in memory, so fetching the metadata of a given key is an efficient operation.

Field Summary

static java.lang.String	BACKUP_DIRECTORY Static variables which define the location of the storage root
static java.lang.String	LOST_AND_FOUND_DIRECTORY
static int	MAX_DIRECTORIES The maximum number of subdirectories in a directory before splitting
static int	MAX_FILES The splitting factor, or the number of files in one directory
static java.lang.String	METADATA_FILENAME
static int	METADATA_SYNC_TIME The amount of time before re-writing the metadata file
static long	PERSISTENCE_MAGIC_NUMBER Static variables defining the layout of the on-disk storage
static long	PERSISTENCE_REVISION_2_0
static long	PERSISTENCE_REVISION_2_1
static long	PERSISTENCE_VERSION_2
static java.lang.String	ZERO_LENGTH_NAME Special placeholder for the file whose name should be zero-length

Constructor Summary

PersistentStorage(IdFactory factory, java.lang.String rootDir, long size, Environment env)
Builds a PersistentStorage given a root directory in which to persist the data.

PersistentStorage(IdFactory factory, java.lang.String name, java.lang.String rootDir, long size, boolean index, Environment env)
Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

PersistentStorage(IdFactory factory, java.lang.String name, java.lang.String rootDir, long size, Environment env)
Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

Method Summary

boolean	exists(Id id) Returns whether or not an object is present in the location id.
void	flush(Continuation c) Method which is used to erase all data stored in the Catalog.
java.io.Serializable	getMetadata(Id id) Returns the metadata associated with the provided object, or null if no metadata exists.
java.lang.String	getName() Gets the name of this instance
void	getObject(Id id, Continuation c) Returns the object identified by the given id.
protected IdRange	getRangeForDirectory(java.io.File dir) Utility function which returns the range of keys which a directory corresponds to.
java.lang.String	getRoot() gets the root directory that the persistence Manager uses
int	getSize() Returns the number of Ids currently stored in the catalog
long	getStorageSize() gets the amount of storage that the persistence Manager uses
long	getTotalSize() Returns the total size of the stored data in bytes.The result is returned via the receiveResult method on the provided Continuation with an Integer representing the size.
void	rename(Id oldId, Id newId, Continuation c) Renames the given object to the new id.
IdSet	scan() Return the objects identified by the given range of ids.
IdSet	scan(IdRange range) Return the objects identified by the given range of ids.
java.util.SortedMap	scanMetadata() Returns a map which contains keys mapping ids to the associated metadata.
java.util.SortedMap	scanMetadata(IdRange range) Returns a map which contains keys mapping ids to the associated metadata.
java.util.SortedMap	scanMetadataValuesHead(java.lang.Object value) Returns the submapping of ids which have metadata less than the provided value.
java.util.SortedMap	scanMetadataValuesNull() Returns the submapping of ids which have metadata null
void	setMetadata(Id id, java.io.Serializable metadata, Continuation c) Updates the metadata stored under the given key to be the provided value.
boolean	setRoot(java.lang.String dir) Sets the root directory that the persistence Manager uses
boolean	setStorageSize(long size) Sets the amount of storage that the persistence Manager uses
void	setTimer(Timer timer) Method which allows the persistence root to schedle an event which will tell it to sync the metadata cached.
void	store(Id id, java.io.Serializable metadata, java.io.Serializable obj, Continuation c) Makes the object persistent to disk and stored permanantly If the object is already persistent, this method will simply update the object's serialized image.
void	unstore(Id id, Continuation c) Request to remove the object from the list of persistend objects.
protected void	writeDirty() Function which writes out all of the dirty metadata files and marks them as clean.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PERSISTENCE_MAGIC_NUMBER

public static final long PERSISTENCE_MAGIC_NUMBER

Static variables defining the layout of the on-disk storage

See Also:

Constant Field Values

PERSISTENCE_VERSION_2

public static final long PERSISTENCE_VERSION_2

See Also:

Constant Field Values

PERSISTENCE_REVISION_2_0

public static final long PERSISTENCE_REVISION_2_0

See Also:

Constant Field Values

PERSISTENCE_REVISION_2_1

public static final long PERSISTENCE_REVISION_2_1

See Also:

Constant Field Values

BACKUP_DIRECTORY

public static final java.lang.String BACKUP_DIRECTORY

Static variables which define the location of the storage root

See Also:

Constant Field Values

LOST_AND_FOUND_DIRECTORY

public static final java.lang.String LOST_AND_FOUND_DIRECTORY

See Also:

Constant Field Values

METADATA_FILENAME

public static final java.lang.String METADATA_FILENAME

See Also:

Constant Field Values

MAX_FILES

public static final int MAX_FILES

The splitting factor, or the number of files in one directory

See Also:

Constant Field Values

MAX_DIRECTORIES

public static final int MAX_DIRECTORIES

The maximum number of subdirectories in a directory before splitting

See Also:

Constant Field Values

METADATA_SYNC_TIME

public static final int METADATA_SYNC_TIME

The amount of time before re-writing the metadata file

See Also:

Constant Field Values

ZERO_LENGTH_NAME

public static final java.lang.String ZERO_LENGTH_NAME

Special placeholder for the file whose name should be zero-length

See Also:

Constant Field Values

Constructor Detail

PersistentStorage

public PersistentStorage(IdFactory factory,
                         java.lang.String rootDir,
                         long size,
                         Environment env)
                  throws java.io.IOException

Builds a PersistentStorage given a root directory in which to persist the data. Uses a default instance name.

Parameters:

factory - The factory to use for creating Ids.

rootDir - The root directory of the persisted disk.

size - the size of the storage in bytes, or -1 for unlimited

Throws:

java.io.IOException

PersistentStorage

public PersistentStorage(IdFactory factory,
                         java.lang.String name,
                         java.lang.String rootDir,
                         long size,
                         Environment env)
                  throws java.io.IOException

Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

Parameters:

factory - The factory to use for creating Ids.

name - the name of this instance

rootDir - The root directory of the persisted disk.

size - the size of the storage in bytes, or -1 for unlimited

Throws:

java.io.IOException

PersistentStorage

public PersistentStorage(IdFactory factory,
                         java.lang.String name,
                         java.lang.String rootDir,
                         long size,
                         boolean index,
                         Environment env)
                  throws java.io.IOException

Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

Parameters:

factory - The factory to use for creating Ids.

name - the name of this instance

rootDir - The root directory of the persisted disk.

size - the size of the storage in bytes, or -1 for unlimited

index - Whether or not to index the objects

Throws:

java.io.IOException

Method Detail

setTimer

public void setTimer(Timer timer)

Method which allows the persistence root to schedle an event which will tell it to sync the metadata cached. Should be called exactly once after the persistence root is created.

Parameters:

timer - The timer to use to schedule the events

rename

public void rename(Id oldId,
                   Id newId,
                   Continuation c)

Renames the given object to the new id. This method is potentially faster than store/cache and unstore/uncache.

Specified by:

rename in interface Catalog

Parameters:

oldId - The id of the object in question.

newId - The new id of the object in question.

c - The command to run once the operation is complete

store

public void store(Id id,
                  java.io.Serializable metadata,
                  java.io.Serializable obj,
                  Continuation c)

Makes the object persistent to disk and stored permanantly If the object is already persistent, this method will simply update the object's serialized image. This is implemented atomically so that this may succeed and store the new object, or fail and leave the previous object intact. This method completes by calling recieveResult() of the provided continuation with the success or failure of the operation.

Specified by:

store in interface Storage

Parameters:

obj - The object to be made persistent.

id - The object's id.

metadata - The object's metadata

c - The command to run once the operation is complete

unstore

public void unstore(Id id,
                    Continuation c)

Request to remove the object from the list of persistend objects. Delete the serialized image of the object from stable storage. If necessary. If the object was not in the cached list in the first place, nothing happens and false is returned. This method also guarantees that the data on disk will remain consistent, even after a crash by performing the delete atomically. This method completes by calling recieveResult() of the provided continuation with the success or failure of the operation.

Specified by:

unstore in interface Storage

Parameters:

id - The object's persistence id

c - The command to run once the operation is complete

exists

public boolean exists(Id id)

Returns whether or not an object is present in the location id.

Specified by:

exists in interface Catalog

Parameters:

id - The id of the object in question.

Returns:

Whether or not an object is present at id.

getMetadata

public java.io.Serializable getMetadata(Id id)

Returns the metadata associated with the provided object, or null if no metadata exists. The metadata must be stored in memory, so this operation is guaranteed to be fast and non-blocking.

Specified by:

getMetadata in interface Catalog

Parameters:

id - The id for which the metadata is needed

Returns:

The metadata, or null of non exists

setMetadata

public void setMetadata(Id id,
                        java.io.Serializable metadata,
                        Continuation c)

Updates the metadata stored under the given key to be the provided value. As this may require a disk access, the requestor must also provide a continuation to return the result to.

Specified by:

setMetadata in interface Catalog

Parameters:

id - The id for the metadata

metadata - The metadata to store

c - The command to run once the operation is complete

getObject

public void getObject(Id id,
                      Continuation c)

Returns the object identified by the given id.

Specified by:

getObject in interface Catalog

Parameters:

id - The id of the object in question.

c - The command to run once the operation is complete

scan

public IdSet scan(IdRange range)

Return the objects identified by the given range of ids. The IdSet returned contains the Ids of the stored objects. The range is partially inclusive, the lower range is inclusive, and the upper exclusive. NOTE: This method blocks so if the behavior of this method changes and uses the disk, this method may be deprecated.

Specified by:

scan in interface Catalog

Parameters:

range - The range to query

Returns:

The idset containg the keys

scan

public IdSet scan()

Return the objects identified by the given range of ids. The IdSet returned contains the Ids of the stored objects. The range is partially inclusive, the lower range is inclusive, and the upper exclusive. NOTE: This method blocks so if the behavior of this method changes and uses the disk, this method may be deprecated.

Specified by:

scan in interface Catalog

Returns:

The idset containg the keys

scanMetadata

public java.util.SortedMap scanMetadata(IdRange range)

Returns a map which contains keys mapping ids to the associated metadata.

Specified by:

scanMetadata in interface Catalog

Parameters:

range - The range to query

Returns:

The map containg the keys

scanMetadata

public java.util.SortedMap scanMetadata()

Returns a map which contains keys mapping ids to the associated metadata.

Specified by:

scanMetadata in interface Catalog

Returns:

The treemap mapping ids to metadata

scanMetadataValuesHead

public java.util.SortedMap scanMetadataValuesHead(java.lang.Object value)

Returns the submapping of ids which have metadata less than the provided value.

Specified by:

scanMetadataValuesHead in interface Catalog

Parameters:

value - The maximal metadata value

Returns:

The submapping

scanMetadataValuesNull

public java.util.SortedMap scanMetadataValuesNull()

Returns the submapping of ids which have metadata null

Specified by:

scanMetadataValuesNull in interface Catalog

Returns:

The submapping

getTotalSize

public long getTotalSize()

Returns the total size of the stored data in bytes.The result is returned via the receiveResult method on the provided Continuation with an Integer representing the size.

Specified by:

getTotalSize in interface Catalog

Parameters:

c - The command to run once the operation is complete

Returns:

The total size, in bytes, of data stored.

getSize

public int getSize()

Returns the number of Ids currently stored in the catalog

Specified by:

getSize in interface Catalog

Returns:

The number of ids in the catalog

flush

public void flush(Continuation c)

Method which is used to erase all data stored in the Catalog. Use this method with care!

Specified by:

flush in interface Catalog

Parameters:

c - The command to run once done

writeDirty

protected void writeDirty()

Function which writes out all of the dirty metadata files and marks them as clean.

getRangeForDirectory

protected IdRange getRangeForDirectory(java.io.File dir)

Utility function which returns the range of keys which a directory corresponds to.

Parameters:

dir - The directory

setRoot

public boolean setRoot(java.lang.String dir)

Sets the root directory that the persistence Manager uses

Parameters:

dir - the String representing the directory to use

Returns:

boolean, true if the operation suceeds false if it doesn't

getRoot

public java.lang.String getRoot()

gets the root directory that the persistence Manager uses

Returns:

String the directory for the root

getStorageSize

public long getStorageSize()

gets the amount of storage that the persistence Manager uses

Returns:

int the amount of storage in bytes allocated for use

setStorageSize

public boolean setStorageSize(long size)

Sets the amount of storage that the persistence Manager uses

Parameters:

size - the amount of storage available to use in bytes

Returns:

boolean, true if the operation suceeds false if it doesn't

getName

public java.lang.String getName()

Gets the name of this instance

Returns:

String the name of the instance