The FreePastry Tutorial.

This tutorial is designed to get you cooking quickly with the FreePastry API and software toolkit.

Version 1.3; August 1, 2005. For FreePastry version 1.4.2. Maintained by Jeff Hoye.

Contents

What You'll Need

Before you begin, you should meet the following requirements.
  1. An intermediate level of knowledge in the Java programming language. The Java Tutorial is a good place to start.
  2. A copy of a current JDK, 1.4.2_06 or later. Sun's J2SE.
  3. A basic understanding of Pastry overlay structure. Read the short overview [pdf][ps] of FreePastry.
  4. Have a working copy of FreePastry-1.4_01 or later. You can simply download the current jar, or download and build the source.

Lesson 0.a

Continuations.

FreePastry and its applications use Continuations in various places, so it is important that you understand what they do and how to use them.

A continuation is similar to a callback, or in java, a Listener, but it is typically used only once. A google search for continuations will show you a variety of uses in computer science, but the primary use for them in FreePastry is to handle network latency, or other types of IO that may block or take a significant amount of time.

To understand why we use continuations, let's look at an alternative style of calls. RMI/RPC style calls are generally blocking calls. Blocking calls work similar to a regular function call. The lookup command in Past (FreePastry's DHT) could look like: 
  public Result lookup(Id id) {
    // blocking network call
  }
But if the network call takes a while to execute due to latency, then your program will block until it completes. In the meantime you could be doing other lookups, but because of this style of call, you are forced to wait, or use a lot of threads. Continuations allow you to "continue" processing when the result arrives, and in the meantime issue other requests.

Let's look at the rice.Continuation interface. 
/**
 * Asynchronously receives the result to a given method call, using
 * the command pattern.
 * 
 * Implementations of this class contain the remainder of a computation
 * which included an asynchronous method call.  When the result to the
 * call becomes available, the receiveResult method on this command
 * is called.
 *
 * @version $Id: index.html,v 1.18 2005/08/10 15:42:30 jeffh Exp $
 *
 * @author Alan Mislove
 * @author Andreas Haeberlen
 */
public interface Continuation {

  /**
   * Called when a previously requested result is now availble.
   *
   * @param result The result of the command.
   */
  public void receiveResult(Object result);

  /**
   * Called when an execption occured as a result of the
   * previous command.
   *
   * @param result The exception which was caused.
   */
  public void receiveException(Exception result);
}
In Past (the DHT that runs on top of FreePastry) the code to do a lookup looks like this: 
  public void lookup(Id id, Continuation command) {
    // non-blocking network call
  }
Note the differences between this call and the alternative approach mentioned before:
  1. The method takes a Continuation.
  2. There is no return value.
When the result arrives, Past will call command.receiveResult() with the response. Inside the continuation, you can complete the task that required the lookup. If instead, an error occurs, Past will call command.receiveException(). Note that because receiveResult() is called with an Object, you must cast the result to what you are expecting from the lookup.

Here is an example implementation of a continuation MyContinuation.java:

First, create the continuation : 
class MyContinuation implements Continuation {
  /**
   * Called when the result arrives.
   */
  public void receiveResult(Object result) {
    PastContent pc = (PastContent)result;
    System.out.println("Received a "+pc);
  }

  /**
   * Called if there is an error.
   */
  public void receiveException(Exception result) {
    System.out.println("There was an error: "+result);      
  }
}
Note that the continuation has 2 methods it must implement: receiveResult() and receiveException(). This continuation will print "Received a blah" when the result comes in. Note that we need to cast the result to what we should be expecting. In the case of Past, we know that the result will be a PastContent. If there is an error, we will print "There was an error: Description of the error"

Here is the code to use our continuation. Note that this code will not run, because Past and Id have not been initialized. (You will learn how to do that in Lesson 7).

Download the code here: TestContinuation.java 
public class TestContinuation {
  public static void main(String[] args) {
    Past past = null; // generated elsewhere
    Id id = null; // generated elsewhere
    
    // create the continuation
    Continuation command = new MyContinuation();    
    
    // make the call with the continuation
    past.lookup(id, command);    
  }
}
This code calls lookup with the continuation. When the result arrives, command.receiveResult() will be called.

Anonymous inner classes

Java hackers love anonymous inner classes, so here is the same code as an anonymous inner class, rather than a seperate class. This is handy because it flows a bit more like a typical blocking program. First do a past.lookup() then System.out.println().

Download the code here: TestContinuationAnon.java 
public class TestContinuationAnon {

  public static void main(String[] args) {
    Past past = null; // generated elsewhere
    Id id = null; // generated elsewhere
    
    // same code as TestContinuation and MyContinuation combined
    past.lookup(id, new Continuation() {
      // will be called if success in the lookup
      public void receiveResult(Object result) {
        PastContent pc = (PastContent)result;
        System.out.println("Received a "+pc);        
      }

      // will be called if failure in the lookup
      public void receiveException(Exception result) {
        System.out.println("There was an error: "+result);      
      }
    });    
  }
}
Click here for more information on Java Anonymous Inner Classes.

Lesson 0.b

Environment

The environment allows us to more easily virtualize pastry inside the same Virtual machine. This is the main new feature of FreePastry 1.4.2. The environment serves FreePastry in 6 key ways: Pastry applications access the Environment from the PastryNode or the commonapi Node (PastryNode implements Node). Generally you can use the same Environment for everything inside your JVM. The two Objects that you will likely construct with an Environment are the PastryNodeFactory and the IdFactory.

How should I get an environment?

The default constructor for an environment looks in the classpath for a params file called freepastry.params. If you compiled the jar with the included ant task, or used the jar from the web, this should work automagically. If you are compiling FreePastry from an IDE (such as eclipse) or from the command line (javac) then you will need to make sure to put the freepastry.params file located in jars/freepastry in your classpath, or use a different contstructor for the Environment.

My JVM won't exit, or How should I get rid of an environment?

The SelectorManager and Processor both have daemon threads that are used to handle communication and processing for the PastryNode. For the JVM to be able to exit, you will need to properly destroy the environment, by calling environment.destroy(). This will call store on the parameters, and destroy the selector/processor threads. Of course, you can also call System.exit(), but that is generally considered bad form.

This should be sufficient information for you to continue with the tutorial. You can learn more in Lesson 10 Advanced Environment.

Lesson 1

Minimal code to create/join a pastry ring.

Download the tutorial file DistTutorial.java.

In this lesson we are going to use Socket transport layer to communicate. The Socket transport layer is a package within the pastry source tree that is used to communicate between nodes using Internet Protocol (IP). It is located in rice.pastry.socket. Socket uses TCP for all messages except liveness which it uses UDP to accomplish. Before we dive into the code lets start with a short vocabulary lesson:

Terms:

Now that you have the proper vocabulary, here is the constructor for the DistTutorial. You can see most of the elements described in the vocabulary. 

  /**
   * This constructor sets up a PastryNode.  It will bootstrap to an 
   * existing ring if it can find one at the specified location, otherwise
   * it will start a new ring.
   * 
   * @param bindport the local port to bind to 
   * @param bootaddress the IP:port of the node to boot from
   * @param env the environment for these nodes
   */
  public DistTutorial(int bindport, InetSocketAddress bootaddress, Environment env) throws Exception {
    
    // Generate the NodeIds Randomly
    NodeIdFactory nidFactory = new RandomNodeIdFactory(env);
    
    // construct the PastryNodeFactory, this is how we use rice.pastry.socket
    PastryNodeFactory factory = new SocketPastryNodeFactory(nidFactory, bindport, env);

    // This will return null if we there is no node at that location
    NodeHandle bootHandle = ((SocketPastryNodeFactory)factory).getNodeHandle(bootaddress);
      
    // construct a node, passing the null boothandle on the first loop will 
    // cause the node to start its own ring
    PastryNode node = factory.newNode(bootHandle);
      
    // the node may require sending several messages to fully boot into the ring
    while(!node.isReady()) {
      // delay so we don't busy-wait
      Thread.sleep(100);
    }
    
    System.out.println("Finished creating new node "+node);    
  }

Let's examine each line:

  1. The arguments that we start with are:
    1. int bindport — the local port to bind to.
    2. InetSocketAddress bootaddress — The address of our bootstrap node.
    3. Environment env — The environment. See lesson 0.b.

    public DistTutorial(int bindport, InetSocketAddress bootaddress, Environment env) throws Exception {
  2. We begin by constructing our NodeIdFactory, which we are going to need to give to our PastryNodeFactory.

    NodeIdFactory nidFactory = new RandomNodeIdFactory(env);
  3. Give the nidFactory to our SocketPastryNodeFactory. Additionally, we need to tell the SocketPastryNodeFactory what port to bind our pastry node to.

    PastryNodeFactory factory = new SocketPastryNodeFactory(nidFactory, bindport, env);
  4. Turn the bootaddress into a NodeHandle. Note that this call blocks, and can take several seconds to complete. It is opening a socket to the specified address. If there is an error, or no PastryNode found at the bootaddress, then getNodeHandle() returns null.

    NodeHandle bootHandle = ((SocketPastryNodeFactory)factory).getNodeHandle(bootaddress);
  5. Finally, create the PastryNode. If bootHandle is null then the factory will start a new ring.

    PastryNode node = factory.newNode(bootHandle);
  6. Even though we don't have an application to run yet, it is important that you are aware of the call to PastryNode.isReady(). This method returns false until the node is fully booted into the ring (which entails establishing his neighbor set i.e. the routing table and the leafset). This simple loop is a typical way to wait for the node to fully boot into the ring.
    while(!node.isReady()) {
  Thread.sleep(100);
}

Before we can run our program lets take a quick look at the main() method. We need to get 1) the local port to bind to, 2) the IP address of the bootstrap node, and 3) the port of the bootstrap node. 

  /**
   * Usage: 
   * java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort
   * example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001
   */
  public static void main(String[] args) throws Exception {
    // Loads pastry settings
    Environment env = new Environment();

    try {
      // the port to use locally
      int bindport = Integer.parseInt(args[0]);
      
      // build the bootaddress from the command line args
      InetAddress bootaddr = InetAddress.getByName(args[1]);
      int bootport = Integer.parseInt(args[2]);
      InetSocketAddress bootaddress = new InetSocketAddress(bootaddr,bootport);
  
      // launch our node!
      DistTutorial dt = new DistTutorial(bindport, bootaddress);
    } catch (Exception e) {
      // remind user how to use
      System.out.println("Usage:"); 
      System.out.println("java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort");
      System.out.println("example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001");
      throw e; 
    } 
  }

Let's examine each line:

  1. This line constructs the Environment. It has the side effect of starting a daemon thread. 
        Environment env = new Environment();
  2. This line parses the first command line argument into an int. 
        int bindport = Integer.parseInt(args[0]);
  3. These lines parse the IP and port and turn them into an InetSocketAddress. 
        InetAddress bootaddr = InetAddress.getByName(args[1]);
    int bootport = Integer.parseInt(args[2]);
    InetSocketAddress bootaddress = new InetSocketAddress(bootaddr,bootport);
  4. Finally we execute our constructor. 
        DistTutorial dt = new DistTutorial(bindport, bootaddress);
  5. We wrap the method with an indication of how to use the program in case the user inputs the wrong args. 
        try {

      ...
      
    } catch (Exception e) {
      // remind user how to use
      System.out.println("Usage:"); 
      System.out.println("java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort");
      System.out.println("example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001");
      throw e; 
    }

Congratulations! You have built code to launch/join a FreePastry ring!
Lesson 2 will aid you in running your code.

Lesson 2

Execute the code to launch your new ring.

This is a short lesson that shows you how to run DistTutorial.java which you created in Lesson 1.

Download the tutorial file DistTutorial.java.

After you compile the code, you can either run multiple nodes on 1 computer (but in separate processes) or if you have multiple computers, you can launch them on different machines as long as the computers can communicate with each other via IP. In other words, it won't work if the computers are behind different firewalls.

Congratulations! You have just launched your first FreePastry ring!
Lesson 3 will show you how to send and receive messages by creating a commonAPI application.

Lesson 3

Write a simple application using the commonAPI.

Download the tutorial files: DistTutorial.java (changed from Lesson 1!!!), MyApp.java, MyMsg.java into a directory called rice/tutorial/lesson3/.

This tutorial will show you how to create and run your first FreePastry application. You will be able to send/receive messages with this application.

Terms:

MyMsg

Let's start by taking a look at your message class. In FreePastry a Message is an Object that is Serializable. The transport layer serializes this object into bytes then sends it through the network. When the message is received, it is deserialized back into an Object and delivered to your application. 

public class MyMsg implements Message {

This class implements rice.p2p.commonapi.Message. Message extends Serializable and has a single method: getPriority(). Let's take a look at that method now.

For now always return Message.LOW_PRIORITY for your messages. It is important to not set application message priority too high, or you may interfere with Pastry's overlay maintenance traffic that keeps the ring functioning properly. 

  public int getPriority() {
    return Message.LOW_PRIORITY;
  }
The "payload" of your message is created by making member variables in your message. Here is the payload for MyMsg. An Id is the commonAPI version of a NodeId, it is also used as the "key" when routing. 
  /**
   * Where the Message came from.
   */
  Id from;
  /**
   * Where the Message is going.
   */
  Id to;
We will create a toString() so that we can print out the message. 
  public String toString() {
    return "MyMsg from "+from+" to "+to;
  }
Finally we have the constructor that loads the payload. 
  public MyMsg(Id from, Id to) {
    this.from = from;
    this.to = to;
  }

MyApp

Now let's take a look at MyApp. MyApp is designed to log output whenever we send or receive a message.

The Endpoint is what we will call on to send messages. 

  protected Endpoint endpoint;

The constructor generates an Endpoint from the node. The instance is designed to allow you to run the same app multiple times on the same node. The apps will not receive each other's messages. You will only be able to send messages to apps that generated endpoints with the same instance string. For most of your apps, you will only run one instance, so just make sure the instance is the same on all nodes. 

  public MyApp(Node node) {
    // We are only going to use one instance of this application on each PastryNode
    this.endpoint = node.registerApplication(this, "myinstance");
  }

Sending a message:

In a Distributed Hash Table, or DHT, you typically want to route to the nearest node to the hash of an object that you are interested in. The commonAPI provides you with Endpoint.route() to accomplish this. This function sends a MyMsg to an id. 

  /**
   * Called to route a message to the id
   */
  public void routeMyMsg(Id id) {
    System.out.println(this+" sending to "+id);    
    Message msg = new MyMsg(endpoint.getId(), id);
    endpoint.route(id, msg, null);
  }
Note that Endpoint.route() takes 3 arguments. They are:
  1. Id—the destination of the message (optional)
  2. Message—the message to send
  3. NodeHandle—a "hint": the node to route to first (optional)
Sometimes you need to send messages to a particular Node in the network, not just to the nearest key. For example, if you need to send messages that are large, they are going to take up a lot of bandwidth; it is more efficient to not route the messages through the overlay, but rather send directly to the node. You can use the Endpoint.route() method to accomplish this by passing a null argument as the Id and the target node's NodeHandle as the "hint" argument. 
  /**
   * Called to directly send a message to the nh
   */
  public void routeMyMsgDirect(NodeHandle nh) {
    System.out.println(this+" sending direct to "+nh);
    Message msg = new MyMsg(endpoint.getId(), nh.getId());
    endpoint.route(null, msg, nh);
  }

Receiving a message:

Simply implement the deliver() method as is specified by the Application interface. 

  /**
   * Called when we receive a message.
   */
  public void deliver(Id id, Message message) {
    System.out.println(this+" received "+message);
  }

For now you don't need to worry about the additional methods in the Application interface.

Congratulations, you have an application. Let's integrate it into DistTutorial.java

Here is the new code we will add to the bottom of the DistTutorial constructor: 
    // construct a new MyApp
    MyApp app = new MyApp(node);
    
    // wait 10 seconds
    Thread.sleep(10000);
    
    // as long as we're not the first node
    if (bootHandle != null) {
      
      // route 10 messages
      for (int i = 0; i < 10; i++) {
        // pick a key at random
        Id randId = nidFactory.generateNodeId();
        
        // send to that key
        app.routeMyMsg(randId);
        
        // wait a sec
        Thread.sleep(1000);
      }

      // wait 10 seconds
      Thread.sleep(10000);
      
      // send directly to my leafset
      LeafSet leafSet = node.getLeafSet();
      
      // this is a typical loop to cover your leafset.  Note that if the leafset
      // overlaps, then duplicate nodes will be sent to twice
      for (int i=-leafSet.ccwSize(); i<=leafSet.cwSize(); i++) {
        if (i != 0) { // don't send to self
          // select the item
          NodeHandle nh = leafSet.get(i);
          
          // send the message directly to the node
          app.routeMyMsgDirect(nh);   
          
          // wait a sec
          Thread.sleep(1000);
        }
      }
    }

First we create the MyApp.

    // construct a new MyApp
    MyApp app = new MyApp(node);

After waiting 10 seconds, let's send some messages, but only if I am not the first node. After all, if I'm the only node in the ring, it is hardly interesting to send messages to myself.

    if (bootHandle != null) {
Loop 10 times. 
      for (int i = 0; i < 10; i++) {
Reuse the RandomNodeIdFactory to generate random keys to route to. 
        Id randId = nidFactory.generateNodeId();
Route. 
        app.routeMyMsg(randId);
Wait a second and repeat. 
        Thread.sleep(1000);
      }

After waiting another 10 seconds, let's send some messages directly to nodes. This section is also going to show you how to access the leafset from the PastryNode. Note that this is a FreePastry specific call, not a commonAPI call.

Get the leafset from the PastryNode:

      LeafSet leafSet = node.getLeafSet();
Iterate over all of the nodes in the leafset. 
      for (int i=-leafSet.ccwSize(); i<=leafSet.cwSize(); i++) {
Don't send to myself. The local node is node zero in the leafset. 
        if (i != 0) { // don't send to self
Extract the nodehandle at that index. 
          NodeHandle nh = leafSet.get(i);
Wait a second and repeat. 
        Thread.sleep(1000);
      }
Send the message. 
          app.routeMyMsgDirect(nh);   
      }
Now if you execute this code twice you should get something like:

(for Node1)

java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson3.DistTutorial 9001 10.9.8.7 9001
:1122933198281:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122933198296:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996])
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xA67C20..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xBF799E..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC4BEE7..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x86ACA9..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x9906E6..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x8F5015..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC20545..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC20545..>

(for Node2)

java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson3.DistTutorial 9002 10.9.8.7 9001
Finished creating new node SocketNodeHandle (<0xDD90C6..>/FOO/10.9.8.7:9002 [5138450490561334965])
MyApp <0xDD90C6..> sending to <0x2E5C63..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x2E5C63..>
MyApp <0xDD90C6..> sending to <0x03045C..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x03045C..>
MyApp <0xDD90C6..> sending to <0xA67C20..>
MyApp <0xDD90C6..> sending to <0xF9C506..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0xF9C506..>
MyApp <0xDD90C6..> sending to <0xBF799E..>
MyApp <0xDD90C6..> sending to <0xC4BEE7..>
MyApp <0xDD90C6..> sending to <0x86ACA9..>
MyApp <0xDD90C6..> sending to <0x41F900..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x41F900..>
MyApp <0xDD90C6..> sending to <0x9906E6..>
MyApp <0xDD90C6..> sending to <0x8F5015..>
MyApp <0xDD90C6..> sending direct to [SNH: <0xDD90C6..> -> <0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996]]
MyApp <0xDD90C6..> sending direct to [SNH: <0xDD90C6..> -> <0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996]]

Congratulations! You have just sent and received your first messages in FreePastry!

Lesson 4

Running multiple nodes in the same JVM.

Download the tutorial files: DistTutorial.java (changed from Lesson 3!!!), MyApp.java, MyMsg.java into a directory called rice/tutorial/lesson4/.

This tutorial will show you how to create and run multiple FreePastry nodes within the same JVM. We will continue to use the Socket transport layer. This tutorial is very simple, all we will be doing is modifying lesson 3 to have multiple nodes.
Let's start with the changes to MyApp. We are going to make it keep a reference to the Node with a member variable named node. It will become apparent later in this tutorial. 

  protected Node node;
In the constructor: 
    this.node = node;
And here is the getter: 
  public Node getNode() {
    return node;
  }
Now we will look at the changes to DistTutorial. We have a member variable apps that will keep track of the list of apps. 
  Vector apps = new Vector();
We modify the constructor to take in how many nodes we wish to create: 
  public DistTutorial(int bindport, InetSocketAddress bootaddress, int numNodes, Environment env) throws Exception {
After constructing the PastryNodeFactory we create a loop to construct the nodes, and add them to our apps Vector. 
    for (int curNode = 0; curNode < numNodes; curNode++) {
	    
	    ...
	    
	    apps.add(app); 
    }
When routing messages, we have created a subloop to do so for each app in the system: 
      // for each app
      Iterator appIterator = apps.iterator();
      while(appIterator.hasNext()) {
        MyApp app = (MyApp)appIterator.next();
        
        ...
        
      }
Note that we eliminated this code, because it is no longer important to not route if we are the bootstrap. 
    // as long as we're not the first node
    if (bootHandle != null) {
In the send direct loop we use our getter MyApp.getNode(). This prevents us from having to remember a parallel data structure for the Nodes. 
    // for each app
    Iterator appIterator = apps.iterator();
    while(appIterator.hasNext()) {
      MyApp app = (MyApp)appIterator.next();
      PastryNode node = (PastryNode)app.getNode();
Other than that, we changed the delay in several locations so the program doesn't take as long to execute. Compile and run this program as before, but make sure to specify some the number of nodes to run as the 4th argument. 
java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson4.DistTutorial 9001 10.9.8.7 9001 10
:1122933905343:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122933905359:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0x93426F..>/FOO/10.9.8.7:9001 [6666451457797044406])
Finished creating new node SocketNodeHandle (<0xAE1B63..>/FOO/10.9.8.7:9002 [8904287830673843378])
Finished creating new node SocketNodeHandle (<0x3E99E3..>/FOO/10.9.8.7:9003 [-1171813548315184530])
Finished creating new node SocketNodeHandle (<0xC8715F..>/FOO/10.9.8.7:9004 [6496343980737954414])
Finished creating new node SocketNodeHandle (<0x688826..>/FOO/10.9.8.7:9005 [-6779179910423314388])
Finished creating new node SocketNodeHandle (<0x640179..>/FOO/10.9.8.7:9006 [3471226224877450778])
Finished creating new node SocketNodeHandle (<0x7A6F99..>/FOO/10.9.8.7:9007 [7101267768505206817])
Finished creating new node SocketNodeHandle (<0xD5DAAE..>/FOO/10.9.8.7:9008 [7899110252170216207])
Finished creating new node SocketNodeHandle (<0x19C832..>/FOO/10.9.8.7:9009 [-4206970387543598705])
Finished creating new node SocketNodeHandle (<0x490C38..>/FOO/10.9.8.7:9010 [-6742572645641651296])
MyApp <0x93426F..> sending to <0x0E510A..>
MyApp <0x19C832..> received MyMsg from <0x93426F..> to <0x0E510A..>
MyApp <0xAE1B63..> sending to <0xE704A5..>
MyApp <0xD5DAAE..> received MyMsg from <0xAE1B63..> to <0xE704A5..>
MyApp <0x3E99E3..> sending to <0x533803..>
MyApp <0x490C38..> received MyMsg from <0x3E99E3..> to <0x533803..>
MyApp <0xC8715F..> sending to <0x779E75..>
MyApp <0x7A6F99..> received MyMsg from <0xC8715F..> to <0x779E75..>
MyApp <0x688826..> sending to <0x3FFFB9..>
MyApp <0x3E99E3..> received MyMsg from <0x688826..> to <0x3FFFB9..>

...

MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0xAE1B63..>/FOO/10.9.8.7:9002 [8904287830673843378]]
MyApp <0xAE1B63..> received MyMsg from <0x93426F..> to <0xAE1B63..>
MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0xC8715F..>/FOO/10.9.8.7:9004 [6496343980737954414]]
MyApp <0xC8715F..> received MyMsg from <0x93426F..> to <0xC8715F..>
MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0xD5DAAE..>/FOO/10.9.8.7:9008 [7899110252170216207]]
MyApp <0xD5DAAE..> received MyMsg from <0x93426F..> to <0xD5DAAE..>
MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0x19C832..>/FOO/10.9.8.7:9009 [-4206970387543598705]]
MyApp <0x19C832..> received MyMsg from <0x93426F..> to <0x19C832..>
MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0x3E99E3..>/FOO/10.9.8.7:9003 [-1171813548315184530]]
MyApp <0x3E99E3..> received MyMsg from <0x93426F..> to <0x3E99E3..>
MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0x490C38..>/FOO/10.9.8.7:9010 [-6742572645641651296]]
MyApp <0x490C38..> received MyMsg from <0x93426F..> to <0x490C38..>

...

Congratulations! You have just created an entire ring inside a single JVM. This should make it much easier to test your application.

Lesson 5

Scheduling tasks on the FreePastry timer.

Download the tutorial files: DistTutorial.java, MyApp.java into a directory called rice/tutorial/lesson5/.

This tutorial will show you how to schedule a task on the FreePastry thread. To be compatible with the commonAPI, the approach is to schedule a message to be sent to yourself at a regular interval.

Let's start with the changes to MyApp.

First a simple message. 

  class MessageToSelf implements Message {
    public int getPriority() {
      return 0;
    }    
  }

Note that due to laziness, I have constructed this Message as an inner class of MyApp. This is fine for what we are going to use it for, but beware that it you will get a NotSerializableException if you attempt to send this message over the wire. While the Message may be serializable it has an implicit reference to MyApp, the containing class. When the message is serialized, FreePastry will also attempt to serialize MyApp which will fail and throw the exception.

Here is the change to the constructor of MyApp, where we schedule the MessageToSelf. The first parameter is the Message that will be sent to MyApp. The second is the delay (in millis) until it is first sent. The third parameter is the period at which the message should be repeated. 

    // Send MessageToSelf every 5 seconds, starting in 3 seconds
    messageToSelfTask = endpoint.scheduleMessage(new MessageToSelf(), 3000, 5000);

As you can see, we have asked FreePastry to deliver the MessageToSelf in 3 seconds, and then continue to deliver it again every 5 seconds. Note that there is also a version of scheduleMessage() that doesn't take the 3rd parameter, and therefore doesn't repeat.

We stored the task as a member variable so that we can cancel it later. 

  CancellableTask messageToSelfTask;

Whenever we receive the MessageToSelf we are just going to print out the current time. However, in your application you could call a method to do routine maintenance, send messages to other nodes etc. 

  public void deliver(Id id, Message message) {
    System.out.println(this+" received "+message);
    if (message instanceof MessageToSelf) {
      // This will get called every 5 seconds, on Pastry's thread.
      // Thus now we can assume we are on Pastry's thread.
      // TODO: whatever... send messages to other nodes? print out status?
      System.out.println("I got the MessageToSelf at time:"+System.currentTimeMillis());
    }
  }

Finally, we make a public method to cancel the task. 

  public void cancelTask() {
    messageToSelfTask.cancel(); 
  }

The DistTutorial.java is based on lesson3, but we removed the code to print the leafset. For this example it is unnecessary to launch multiple pastry nodes, so we will only launch one. We added a bit of code after constructing the MyApp to cancel the task after 15 seconds. 

    // construct a new MyApp
    MyApp app = new MyApp(node);
    
    // wait 15 seconds
    Thread.sleep(15000);
    
    // cancel the task
    app.cancelTask();

Ok, time to run it! Your output should resemble the following: 

java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson5.DistTutorial 9001 localhost 9001
:1122932166578:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122932166578:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0x3C1EEA..>/FOO/10.9.8.7:9001 [9038308767327354496])
MyApp <0x3C1EEA..> received rice.tutorial.lesson5.MyApp$MessageToSelf@d70d7a
I got the MessageToSelf at time:1115228553359
MyApp <0x3C1EEA..> received rice.tutorial.lesson5.MyApp$MessageToSelf@d70d7a
I got the MessageToSelf at time:1115228558359
MyApp <0x3C1EEA..> received rice.tutorial.lesson5.MyApp$MessageToSelf@d70d7a
I got the MessageToSelf at time:1115228563359

Congratulations! Now you can schedule and cancel tasks without launching your own thread.

Lesson 6

Introducing Scribe.

Download the tutorial files: MyScribeClient.java, MyScribeContent.java, ScribeTutorial.java into a directory called rice/tutorial/lesson6/.

Scribe is an application that allows you to subscribe to groups and publish messages to that group. This tutorial will show you how to get scribe up and running. You will learn how to do the following:

Lesson 9: Advanced Scribe: Policies, will show you how to control tree formation and anycast selection.

Terms:

Creating a topic.

This is fairly straightforward, but each node will have to do this for each topic of interest. 
  Topic myTopic = new Topic(new PastryIdFactory(), "example topic");
This constructs a topic with the "common name" of "example topic". It uses the PastryIdFactory to generate an appropriate Id for this topic.

Creating a ScribeClient.

Let's take a look at MyScribeClient. We are only going to subscribe to a single topic. We are going to publish content every 5 seconds using the FreePastry timer. See Lesson 5 for more details on the timer. The client is also going to implement rice.p2p.commonapi.Application in addition to rice.p2p.scribe.ScribeClient. This will allow us to send and receive non-scribe messages should this be important. Specifically it will allow us to receive messages queued on the timer.

Here's the constructor and some member variables: 
  Scribe myScribe;
  Topic myTopic;
  protected Endpoint endpoint;
  public MyScribeClient(PastryNode node) {
    // you should recognize this from lesson 3
    this.endpoint = node.registerApplication(this, "myinstance");
    // construct Scribe
    myScribe = new ScribeImpl(node,"lesson6instance");
    // construct the topic
    myTopic = new Topic(new PastryIdFactory(), "example topic");
    System.out.println("myTopic = "+myTopic);
  }
The only thing that should be new here is the construction of the ScribeImpl. The instance name "lesson6instance" allows you to remain independent of other applications running on the same ring who also use scribe. They will have their own instance of Scribe that won't be confused with your instance.
In this example, one of the nodes is going to publish content every 5 seconds. We use the timer pattern described in Lesson 5. Each time we are going to send 1 multicast, and 1 anycast. 
  class PublishContent implements Message {
    public int getPriority() {
      return 0;
    }
  }
  public void startPublishTask() {
    publishTask = endpoint.scheduleMessage(new PublishContent(), 5000, 5000);    
  }
  public void deliver(Id id, Message message) {
    if (message instanceof PublishContent) {
      sendMulticast(); 
      sendAnycast();
    }
  }
In case you forgot, the PublishContent is similar to Lesson 5's MessageToSelf. The startPublishTask() method schedules this to be delivered locally every 5 seconds. The deliver(id,message) method calls sendMulticast() when the PublishMethod is received. In your application you will likely have some other event that causes content to be published. We will look at sendMulticast() and sendAnycast() shortly.

Subscribing to a group.

Subscribing is very easy. Just call Scribe.subscribe() and provide the topic, and your client. 
  public void subscribe() {
    myScribe.subscribe(myTopic, this); 
  }

Multicasting content.

First, we need some content to send. MyScribeContent implements ScribeContent and takes a NodeHandle sender, and an int sequence number. These are just so the output of the program is more interesting. 
public class MyScribeContent implements ScribeContent {
  NodeHandle from;
  int seq;
 
  public MyScribeContent(NodeHandle from, int seq) {
    this.from = from;
    this.seq = seq;
  }

  public String toString() {
    return "MyScribeContent #"+seq+" from "+from;
  }  
}
To send the content, simply construct the message, then call Scribe.publish(). You give it the topic and the message. The rest of this function is just to print output and update the sequence number. 
  public void sendMulticast() {
    System.out.println("Node "+endpoint.getLocalNodeHandle()+" broadcasting "+seqNum);
    MyScribeContent myMessage = new MyScribeContent(endpoint.getLocalNodeHandle(), seqNum);
    myScribe.publish(myTopic, myMessage); 
    seqNum++;
  }

Receiving content.

Receiving content is as easy as any other p2p application. The method signature is only slightly different: 
  public void deliver(Topic topic, ScribeContent content) {
    System.out.println("MyScribeClient.deliver("+topic+","+content+")");
  }
All we are doing here is printing output to stdout.

Anycasting.

Anycast will get called on your clients until one returns true. This occurs on a call to ScribeClient.anycast(). To make this interesting, we're going to only accept the message 1/3 of the time, randomly. Your application will ususally want to do something more interesting, such as see if a requested resource is available.
Here is sendAnycast() which is nearly identical to sendMulticast() except for the call to Scribe.anycast() instead of Scribe.publish(). 
  public void sendAnycast() {
    System.out.println("Node "+endpoint.getLocalNodeHandle()+" anycasting "+seqNum);
    MyScribeContent myMessage = new MyScribeContent(endpoint.getLocalNodeHandle(), seqNum);
    myScribe.anycast(myTopic, myMessage); 
    seqNum++;
  }
Here is the code that only accepts the anycast 1/3 of the time. This will allow us to see that the anycast message can be rejected and sent elsewhere. 
  public boolean anycast(Topic topic, ScribeContent content) {
    boolean returnValue = rng.nextInt(3) == 0;
    System.out.println("MyScribeClient.anycast("+topic+","+content+"):"+returnValue);
    return returnValue;
  }

Examining the tree.

Lastly we have code to introspect the tree. Note that it is only easy to print out the entire tree because we are running all of the nodes in the same VM and have global information. It is significantly more difficult to print the scribe tree in an actual distributed environment, as scribe does not provide this information automatically. Furthermore this information could change rapidly as nodes join and leave.

Because we are only going to run the application with a small number of nodes, the tree will most likely be only 1 level deep. (The root being level 0). However, this will show you how to print out your parent and children.

The first thing to note is that we have 3 accessor methods at the bottom of MyScribeContent: 
  public boolean isRoot() {
    return myScribe.isRoot(myTopic);
  }
  
  public NodeHandle getParent() {
    return myScribe.getParent(myTopic); 
  }
  
  public NodeHandle[] getChildren() {
    return myScribe.getChildren(myTopic); 
  }
Note that these simply call through to the same method on myScribe, with the correct topic.

This code can be found in ScribeTutorial.java

printTree() does the following:
  1. Create a table mapping NodeHandle to MyScribeClient.
  2. Recursively traverse the tree to the root, using the helper: getRoot().
  3. Recursively traverse the tree down from the root, depth first and print the nodes using the helper: recursivelyPrintChildren().
  public static void printTree(Vector apps) {
    // build a hashtable of the apps, keyed by nodehandle
    Hashtable appTable = new Hashtable();
    Iterator i = apps.iterator();
    while (i.hasNext()) {
      MyScribeClient app = (MyScribeClient)i.next();
      appTable.put(app.endpoint.getLocalNodeHandle(), app);
    }
    NodeHandle seed = ((MyScribeClient)apps.get(0)).endpoint.getLocalNodeHandle();
    
    // get the root 
    NodeHandle root = getRoot(seed, appTable);
    
    // print the tree from the root down
    recursivelyPrintChildren(root, 0, appTable);
  }
  1. getRoot() looks up the client for the seed handle from the appTable.
  2. If the seed is the root, it is returned.
  3. Otherwise, it calls getRoot() on the parent.
  public static NodeHandle getRoot(NodeHandle seed, Hashtable appTable) {
    MyScribeClient app = (MyScribeClient)appTable.get(seed);
    if (app.isRoot()) return seed;
    NodeHandle nextSeed = app.getParent();
    return getRoot(nextSeed, appTable);
  }
  1. recursivelyPrintChildren() prints the curNode with appropriate whitespace based on the depth in the tree.
  2. Then calls recursivelyPrintChildren() on all children (if it has any)
  public static void recursivelyPrintChildren(NodeHandle curNode, int recursionDepth, Hashtable appTable) {
    // print self at appropriate tab level
    String s = "";
    for (int numTabs = 0; numTabs < recursionDepth; numTabs++) {
      s+="  "; 
    }
    s+=curNode.getId().toString();
    System.out.println(s);
    
    // recursively print all children
    MyScribeClient app = (MyScribeClient)appTable.get(curNode);
    NodeHandle[] children = app.getChildren();
    for (int curChild = 0; curChild < children.length; curChild++) {
      recursivelyPrintChildren(children[curChild], recursionDepth+1, appTable);
    }    
  }

Initializing the apps.

The majority of the ScribeTutorial is identical to Lesson 4 where we ran multiple nodes within the same JVM. The last part is listed below. On each app, we call subscribe(), and on the first one we call startPublishTask(). After that we wait a few seconds then print the tree. 
    Iterator i = apps.iterator();    
    MyScribeClient app = (MyScribeClient)i.next();
    app.subscribe();
    app.startPublishTask();
    while(i.hasNext()) {
      app = (MyScribeClient)i.next();
      app.subscribe();
    }
    
    Thread.sleep(3000);
    printTree(apps);

Execution.

The parameters are identical to those in Lesson 4:
  1. Local bind port.
  2. Bootstrap host. (the local host address)
  3. Bootstrap port. (usually whatever you passed in the first arg)
  4. The number of nodes to launch.
Your output will resemble: 
java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson6.ScribeTutorial 9001 10.9.8.7 9001 10
:1122932166578:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122932166578:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189])
myTopic = [TOPIC <0x19A8F5..>]
Finished creating new node SocketNodeHandle (<0x8904D9..>/FOO/10.9.8.7:9002 [4410881318286179245])
myTopic = [TOPIC <0x19A8F5..>]
Finished creating new node SocketNodeHandle (<0x281817..>/FOO/10.9.8.7:9003 [1144941711194723161])

...

<0x281817..>
  <0x8904D9..>
  <0x061BB8..>
  <0x85DA64..>
  <0x489BCB..>
  <0x0A9FCC..>
  <0x39CE29..>
  <0xA20DF8..>
  <0x7D350E..>
  <0xCF76F1..>
Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] broadcasting 0
Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] anycasting 1
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #1 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #1 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):true
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0xA20DF8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0xCF76F1..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x0A9FCC..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x7D350E..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x85DA64..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x061BB8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] broadcasting 2
Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] anycasting 3
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #3 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #3 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):true
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0xA20DF8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0xCF76F1..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x0A9FCC..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x7D350E..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x85DA64..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x061BB8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] broadcasting 4
Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] anycasting 5
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0xA20DF8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0xCF76F1..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x0A9FCC..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x7D350E..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x85DA64..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x061BB8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):true
Note that the tree is only 1 level deep.
Note how each publish message is delivered to each node in the group.
Note that anycast is called on different nodes until a node returns true.

Congratulations! You have built and run your first scribe application!

Lesson 7

Past &mdash FreePastry's DHT.

Download the tutorial files: MyPastContent.java, PastTutorial.java into a directory called rice/tutorial/lesson7/.

Past is FreePastry's Distributed Hash Table (DHT). This tutorial will show you how to get scribe up and running. You will learn how to do the following:

Terms:

Create a PastContent.

Unlike Scribe, you do not need to write a client to utilize Past. However, you do need to create a content object to put/get. This is called a PastContent. The easiest way to create a PastContent is to extend rice.p2p.past.ContentHashPastContent. Here is the code for MyPastContent. 
public class MyPastContent extends ContentHashPastContent {  
  /**
   * Store the content.
   * 
   * Note that this class is Serializable, so any non-transient field will 
   * automatically be stored to to disk.
   */
  String content;
    
  /**
   * Takes an environment for the timestamp
   * An IdFactory to generate the hash
   * The content to be stored.
   * 
   * @param idf to generate a hash of the content
   * @param content to be stored
   */
  public MyPastContent(Id id, String content) {
    super(id);
    this.content = content;
  }
  
  /**
   * A descriptive toString()
   */
  public String toString() {
    return "MyPastContent ["+content+"]";
  }
}
Note that the super constructor needs an Id. Other than that the class must be serializable.

Create a Past instance.

Now on to the PastTutorial. The first part of PastTutorial looks identical to ScribeTutorial. The interesting part is creating the Past application. 
      // used for generating PastContent object Ids.
      // this implements the "hash function" for our DHT
      PastryIdFactory idf = new rice.pastry.commonapi.PastryIdFactory(env);
      
      // create a different storage root for each node
      String storageDirectory = "./storage"+node.getId().hashCode();

      // create the persistent part
      Storage stor = new PersistentStorage(idf, storageDirectory, 4 * 1024 * 1024, node
          .getEnvironment());
      Past app = new PastImpl(node, new StorageManagerImpl(idf, stor, new LRUCache(
          new MemoryStorage(idf), 512 * 1024, node.getEnvironment())), 3, "");
Recall the PastryIdFactory is basically a hash funciton that produces Ids. The rest of the code sets up a typical PastImpl. It uses PersistentStorage for the primary store, and an LRUCache backed by a MemoryStorage as the cache.

Here is an overview of what is going on.

PastImpl needs the following: StorageManagerImpl (the StorageManager) needs the following: The PersistentStorage needs the following: The LRUCache needs the following:

Put an object in Past.

Now lets store some data. The loop around this code is so we can later look up the data that was stored. 
      // these variables are final so that the continuation can access them
      final String s = "test" + env.getRandomSource().nextInt();
      
      // build the past content
      final PastContent myContent = new MyPastContent(localFactory.buildId(s), s);
    
      storedKey[ctr] = myContent.getId();
      
      // pick a random past appl on a random node
      Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));
      System.out.println("Inserting " + myContent + " at node "+p.getLocalNodeHandle());
      
      // insert the data
      p.insert(myContent, new Continuation() {
        // the result is an Array of Booleans for each insert
        public void receiveResult(Object result) {          
          Boolean[] results = ((Boolean[]) result);
          int numSuccessfulStores = 0;
          for (int ctr = 0; ctr < results.length; ctr++) {
            if (results[ctr].booleanValue()) 
              numSuccessfulStores++;
          }
          System.out.println(myContent + " successfully stored at " + 
              numSuccessfulStores + " locations.");
        }
  
        public void receiveException(Exception result) {
          System.out.println("Error storing "+myContent);
          result.printStackTrace();
        }
      });
Note that we call Past.insert() with our newly created MyPastContent object, and a Continuation. Past.insert() calls receiveResult() on the continuation with a Boolean[] that represents the success/failure of the individual stores for this object in the ring. Our continuation prints how many times this node was stored. Because our replication factor is 3, it should be stored 4 times (primary+3 replicas). If an error occurs it will call receiveException().

Get an object from Past.

We choose a random application and call Past.lookup() 
      final Id lookupKey = storedKey[ctr];
      
      // pick a random past appl on a random node
      Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));

      System.out.println("Looking up " + lookupKey + " at node "+p.getLocalNodeHandle());
      p.lookup(lookupKey, new Continuation() {
        public void receiveResult(Object result) {
          System.out.println("Successfully looked up " + result + " for key "+lookupKey+".");
        }
  
        public void receiveException(Exception result) {
          System.out.println("Error looking up "+lookupKey);
          result.printStackTrace();
        }
      });

A failed Get.

Now let's see what happen if the Past doesn't have the requested content. Here's a hint, it calls receiveResult() with a null result. 
    final Id bogusKey = localFactory.buildId("bogus");

...

    System.out.println("Looking up bogus key " + bogusKey + " at node "+p.getLocalNodeHandle());
    p.lookup(bogusKey, new Continuation() {
      public void receiveResult(Object result) {
        System.out.println("Successfully looked up " + result + " for key "+bogusKey+".  Notice that the result is null.");
      }

      public void receiveException(Exception result) {
        System.out.println("Error looking up "+bogusKey);
        result.printStackTrace();
      }
    });
When you run the code, your output will resemble: 
java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson7.PastTutorial 9001 10.9.8.7 9001 5
:1122487192406:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122487192421:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116])
Finished creating new node SocketNodeHandle (<0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790])
Finished creating new node SocketNodeHandle (<0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501])
Finished creating new node SocketNodeHandle (<0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085])
Finished creating new node SocketNodeHandle (<0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041])
Storing 5 keys
Inserting MyPastContent [test1382830848] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
Inserting MyPastContent [test-618483902] at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
Inserting MyPastContent [test482928483] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
Inserting MyPastContent [test-919775810] at node [SNH: <0x4AE14A..> -> <0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041]]
Inserting MyPastContent [test1099145466] at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
MyPastContent [test1382830848] successfully stored at 4 locations.
MyPastContent [test-618483902] successfully stored at 4 locations.
MyPastContent [test482928483] successfully stored at 4 locations.
MyPastContent [test-919775810] successfully stored at 4 locations.
MyPastContent [test1099145466] successfully stored at 4 locations.
Looking up the 5 keys
Looking up <0x9515FC..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
Looking up <0xB987B7..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
Looking up <0x999A32..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
Looking up <0xC9B06E..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
Looking up <0xC2EF27..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
Successfully looked up MyPastContent [test1099145466] for key <0xC2EF27..>.
Successfully looked up MyPastContent [test-618483902] for key <0xB987B7..>.
Successfully looked up MyPastContent [test482928483] for key <0x999A32..>.
Successfully looked up MyPastContent [test-919775810] for key <0xC9B06E..>.
Successfully looked up MyPastContent [test1382830848] for key <0x9515FC..>.
Looking up a bogus key
Looking up bogus key <0x216ACE..> at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
Successfully looked up null for key <0x216ACE..>.  Notice that the result is null.
Also 5 directories were created, each containing a FreePastry-Storage-Root subfolder: 
ls storage*
storage-1404778543:
FreePastry-Storage-Root

storage-1501429183:
FreePastry-Storage-Root

storage-1862004272:
FreePastry-Storage-Root

storage2042841361:
FreePastry-Storage-Root

storage848913672:
FreePastry-Storage-Root
Pastry tutorial version 1.3.     Last updated August 1, 2005.     For FreePastry-1.4.2.     Maintained by Jeff Hoye