---

Past

FreePastry's DHT.

Download the tutorial files: MyPastContent.java, PastTutorial.java into a directory called rice/tutorial/past/.

Past is FreePastry's Distributed Hash Table (DHT). This tutorial will show you how to get scribe up and running. You will learn how to do the following:

• Create a PastContent.
• Create a Past instance.
• Put an object in Past.
• Get an object from Past.
• A failed Get.

Terms:

• Past—A large-scale, peer-to-peer archival storage facility. Past is FreePastry's DHT. It has insert() and lookup() operations.
• Content—A possible value of the DHT.
• IdFactory—A hash function. An IdFactory implements a hash function to build Ids that are compatable with pastry.
• PastryIdFactory—A commonly used IdFactory in Pastry. The PastryIdFactory uses SHA1 as it's underlying hash function.
• Storage—provides a local storage service for example, a persistence storage service or an in-memory storage service
• MemoryStorage—an implementation of Storage which provides in-memory storage. This class is specifically not designed to provide persistent storage.
• PersistentStorage—an implementation of Storage which provides persistent storage to disk.
• Cache—finite temporary storage for rapid access.
• LRUCache—implemetation of a least-recently-used (LRU) cache.
• StorageManager—a storage attached to a cache. Objects inserted and retrieved from the storage are auto-matically cached, to speed up future accesses.

Create a PastContent.

Unlike Scribe, you do not need to write a client to utilize Past. However, you do need to create a content object to put/get. This is called a PastContent. The easiest way to create a PastContent is to extend rice.p2p.past.ContentHashPastContent. Here is the code for MyPastContent.

public class MyPastContent extends ContentHashPastContent {  
  /**
   * Store the content.
   * 
   * Note that this class is Serializable, so any non-transient field will 
   * automatically be stored to to disk.
   */
  String content;
    
  /**
   * Takes an environment for the timestamp
   * An IdFactory to generate the hash
   * The content to be stored.
   * 
   * @param idf to generate a hash of the content
   * @param content to be stored
   */
  public MyPastContent(Id id, String content) {
    super(id);
    this.content = content;
  }
  
  /**
   * A descriptive toString()
   */
  public String toString() {
    return "MyPastContent ["+content+"]";
  }
}

Note that the super constructor needs an Id. Other than that the class must be serializable.

Create a Past instance.

Now on to the PastTutorial. The first part of PastTutorial looks identical to ScribeTutorial. The interesting part is creating the Past application.

      // used for generating PastContent object Ids.
      // this implements the "hash function" for our DHT
      PastryIdFactory idf = new rice.pastry.commonapi.PastryIdFactory(env);
      
      // create a different storage root for each node
      String storageDirectory = "./storage"+node.getId().hashCode();

      // create the persistent part
      Storage stor = new PersistentStorage(idf, storageDirectory, 4 * 1024 * 1024, node
          .getEnvironment());
      Past app = new PastImpl(node, new StorageManagerImpl(idf, stor, new LRUCache(
          new MemoryStorage(idf), 512 * 1024, node.getEnvironment())), 3, "");

Recall the PastryIdFactory is basically a hash funciton that produces Ids. The rest of the code sets up a typical PastImpl. It uses PersistentStorage for the primary store, and an LRUCache backed by a MemoryStorage as the cache.

Here is an overview of what is going on.

PastImpl needs the following:

• The Node
• StorageManager
• The number of Replicas to maintain — this is a parameter that you can tune to adjust overhead vs. chance of data-loss
• An instance name

StorageManagerImpl (the StorageManager) needs the following:

• An IdFactory
• A storage — we'll use the PersistentStorage
• A cache — we'll use the LRUCache

The PersistentStorage needs the following:

• An IdFactory
• The directory to put the storage root. — Note that we need a different storage root for each node, otherwise they will use the same one. This is only necessary because we are running multiple nodes in the same JVM.
• The max size (in bytes) — 1 Meg
• The environment

The LRUCache needs the following:

• A storage — we'll use a MemoryStorage
• The max size (in bytes) — 0.5 Meg
• The environment

Put an object in Past.

Now lets store some data. The loop around this code is so we can later look up the data that was stored.

      // these variables are final so that the continuation can access them
      final String s = "test" + env.getRandomSource().nextInt();
      
      // build the past content
      final PastContent myContent = new MyPastContent(localFactory.buildId(s), s);
    
      storedKey[ctr] = myContent.getId();
      
      // pick a random past appl on a random node
      Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));
      System.out.println("Inserting " + myContent + " at node "+p.getLocalNodeHandle());
      
      // insert the data
      p.insert(myContent, new Continuation() {
        // the result is an Array of Booleans for each insert
        public void receiveResult(Object result) {          
          Boolean[] results = ((Boolean[]) result);
          int numSuccessfulStores = 0;
          for (int ctr = 0; ctr < results.length; ctr++) {
            if (results[ctr].booleanValue()) 
              numSuccessfulStores++;
          }
          System.out.println(myContent + " successfully stored at " + 
              numSuccessfulStores + " locations.");
        }
  
        public void receiveException(Exception result) {
          System.out.println("Error storing "+myContent);
          result.printStackTrace();
        }
      });

Note that we call Past.insert() with our newly created MyPastContent object, and a Continuation. Past.insert() calls receiveResult() on the continuation with a Boolean[] that represents the success/failure of the individual stores for this object in the ring. Our continuation prints how many times this node was stored. Because our replication factor is 3, it should be stored 4 times (primary+3 replicas). If an error occurs it will call receiveException().

Get an object from Past.

We choose a random application and call Past.lookup()

      final Id lookupKey = storedKey[ctr];
      
      // pick a random past appl on a random node
      Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));

      System.out.println("Looking up " + lookupKey + " at node "+p.getLocalNodeHandle());
      p.lookup(lookupKey, new Continuation() {
        public void receiveResult(Object result) {
          System.out.println("Successfully looked up " + result + " for key "+lookupKey+".");
        }
  
        public void receiveException(Exception result) {
          System.out.println("Error looking up "+lookupKey);
          result.printStackTrace();
        }
      });

A failed Get.

Now let's see what happen if the Past doesn't have the requested content. Here's a hint, it calls receiveResult() with a null result.

    final Id bogusKey = localFactory.buildId("bogus");

...

    System.out.println("Looking up bogus key " + bogusKey + " at node "+p.getLocalNodeHandle());
    p.lookup(bogusKey, new Continuation() {
      public void receiveResult(Object result) {
        System.out.println("Successfully looked up " + result + " for key "+bogusKey+".  Notice that the result is null.");
      }

      public void receiveException(Exception result) {
        System.out.println("Error looking up "+bogusKey);
        result.printStackTrace();
      }
    });

To use past, you need to include 2 additional jars:xmlpull_1_1_3_4a.jar,xpp3-1.1.3.4d_b2.jar. You can find these in the lib directory in the FreePastry source distribution. When you run the code, your output will resemble:

java -cp .:FreePastry-@freepastry_version@.jar:xmlpull_1_1_3_4a.jar:xpp3-1.1.3.4d_b2.jar rice.tutorial.past.PastTutorial 9001 10.9.8.7 9001 5
:1122487192406:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122487192421:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116])
Finished creating new node SocketNodeHandle (<0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790])
Finished creating new node SocketNodeHandle (<0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501])
Finished creating new node SocketNodeHandle (<0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085])
Finished creating new node SocketNodeHandle (<0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041])
Storing 5 keys
Inserting MyPastContent [test1382830848] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
Inserting MyPastContent [test-618483902] at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
Inserting MyPastContent [test482928483] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
Inserting MyPastContent [test-919775810] at node [SNH: <0x4AE14A..> -> <0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041]]
Inserting MyPastContent [test1099145466] at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
MyPastContent [test1382830848] successfully stored at 4 locations.
MyPastContent [test-618483902] successfully stored at 4 locations.
MyPastContent [test482928483] successfully stored at 4 locations.
MyPastContent [test-919775810] successfully stored at 4 locations.
MyPastContent [test1099145466] successfully stored at 4 locations.
Looking up the 5 keys
Looking up <0x9515FC..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
Looking up <0xB987B7..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
Looking up <0x999A32..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
Looking up <0xC9B06E..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
Looking up <0xC2EF27..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
Successfully looked up MyPastContent [test1099145466] for key <0xC2EF27..>.
Successfully looked up MyPastContent [test-618483902] for key <0xB987B7..>.
Successfully looked up MyPastContent [test482928483] for key <0x999A32..>.
Successfully looked up MyPastContent [test-919775810] for key <0xC9B06E..>.
Successfully looked up MyPastContent [test1382830848] for key <0x9515FC..>.
Looking up a bogus key
Looking up bogus key <0x216ACE..> at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
Successfully looked up null for key <0x216ACE..>.  Notice that the result is null.

Also 5 directories were created, each containing a FreePastry-Storage-Root subfolder:

ls storage*
storage-1404778543:
FreePastry-Storage-Root

storage-1501429183:
FreePastry-Storage-Root

storage-1862004272:
FreePastry-Storage-Root

storage2042841361:
FreePastry-Storage-Root

storage848913672:
FreePastry-Storage-Root

---