The FreePastry Tutorial.

This tutorial is designed to get you cooking quickly with the FreePastry API and software toolkit.

Version @tutorial_version@; @tutorial_date@. For FreePastry version @freepastry_version@. Maintained by @maintainer@.

Contents


What You'll Need

Before you begin, you should meet the following requirements.
  1. An intermediate level of knowledge in the Java programming language. The Java Tutorial is a good place to start.
  2. A copy of a current JDK, 1.4.2_06 or later. Sun's J2SE.
  3. A basic understanding of Pastry overlay structure. Read the short overview [pdf][ps] of FreePastry.
  4. Have a working copy of FreePastry-1.4_01 or later. You can simply download the current jar, or download and build the source.

Lesson 0.a

Continuations.

FreePastry and its applications use Continuations in various places, so it is important that you understand what they do and how to use them.

A continuation is similar to a callback, or in java, a Listener, but it is typically used only once. A google search for continuations will show you a variety of uses in computer science, but the primary use for them in FreePastry is to handle network latency, or other types of IO that may block or take a significant amount of time.

To understand why we use continuations, let's look at an alternative style of calls. RMI/RPC style calls are generally blocking calls. Blocking calls work similar to a regular function call. The lookup command in Past (FreePastry's DHT) could look like:
  public Result lookup(Id id) {
    // blocking network call
  }
But if the network call takes a while to execute due to latency, then your program will block until it completes. In the meantime you could be doing other lookups, but because of this style of call, you are forced to wait, or use a lot of threads. Continuations allow you to "continue" processing when the result arrives, and in the meantime issue other requests.

Let's look at the rice.Continuation interface.
/**
 * Asynchronously receives the result to a given method call, using
 * the command pattern.
 * 
 * Implementations of this class contain the remainder of a computation
 * which included an asynchronous method call.  When the result to the
 * call becomes available, the receiveResult method on this command
 * is called.
 *
 * @version $Id: index.html 2716 2005-08-10 17:42:38 +0200 (Wed, 10 Aug 2005) jeffh $
 *
 * @author Alan Mislove
 * @author Andreas Haeberlen
 */
public interface Continuation {

  /**
   * Called when a previously requested result is now availble.
   *
   * @param result The result of the command.
   */
  public void receiveResult(Object result);

  /**
   * Called when an execption occured as a result of the
   * previous command.
   *
   * @param result The exception which was caused.
   */
  public void receiveException(Exception result);
}
In Past (the DHT that runs on top of FreePastry) the code to do a lookup looks like this:
  public void lookup(Id id, Continuation command) {
    // non-blocking network call
  }
Note the differences between this call and the alternative approach mentioned before:
  1. The method takes a Continuation.
  2. There is no return value.
When the result arrives, Past will call command.receiveResult() with the response. Inside the continuation, you can complete the task that required the lookup. If instead, an error occurs, Past will call command.receiveException(). Note that because receiveResult() is called with an Object, you must cast the result to what you are expecting from the lookup.

Here is an example implementation of a continuation MyContinuation.java:

First, create the continuation :
class MyContinuation implements Continuation {
  /**
   * Called when the result arrives.
   */
  public void receiveResult(Object result) {
    PastContent pc = (PastContent)result;
    System.out.println("Received a "+pc);
  }

  /**
   * Called if there is an error.
   */
  public void receiveException(Exception result) {
    System.out.println("There was an error: "+result);      
  }
}
Note that the continuation has 2 methods it must implement: receiveResult() and receiveException(). This continuation will print "Received a blah" when the result comes in. Note that we need to cast the result to what we should be expecting. In the case of Past, we know that the result will be a PastContent. If there is an error, we will print "There was an error: Description of the error"

Here is the code to use our continuation. Note that this code will not run, because Past and Id have not been initialized. (You will learn how to do that in Lesson 7).

Download the code here: TestContinuation.java
public class TestContinuation {
  public static void main(String[] args) {
    Past past = null; // generated elsewhere
    Id id = null; // generated elsewhere
    
    // create the continuation
    Continuation command = new MyContinuation();    
    
    // make the call with the continuation
    past.lookup(id, command);    
  }
}
This code calls lookup with the continuation. When the result arrives, command.receiveResult() will be called.

Anonymous inner classes

Java hackers love anonymous inner classes, so here is the same code as an anonymous inner class, rather than a seperate class. This is handy because it flows a bit more like a typical blocking program. First do a past.lookup() then System.out.println().

Download the code here: TestContinuationAnon.java
public class TestContinuationAnon {

  public static void main(String[] args) {
    Past past = null; // generated elsewhere
    Id id = null; // generated elsewhere
    
    // same code as TestContinuation and MyContinuation combined
    past.lookup(id, new Continuation() {
      // will be called if success in the lookup
      public void receiveResult(Object result) {
        PastContent pc = (PastContent)result;
        System.out.println("Received a "+pc);        
      }

      // will be called if failure in the lookup
      public void receiveException(Exception result) {
        System.out.println("There was an error: "+result);      
      }
    });    
  }
}
Click here for more information on Java Anonymous Inner Classes.

Lesson 0.b

Environment

The environment allows us to more easily virtualize pastry inside the same Virtual machine. This is the main new feature of FreePastry 1.4.2. The environment serves FreePastry in 6 key ways: Pastry applications access the Environment from the PastryNode or the commonapi Node (PastryNode implements Node). Generally you can use the same Environment for everything inside your JVM. The two Objects that you will likely construct with an Environment are the PastryNodeFactory and the IdFactory.

How should I get an environment?

The default constructor for an environment looks in the classpath for a params file called freepastry.params. If you compiled the jar with the included ant task, or used the jar from the web, this should work automagically. If you are compiling FreePastry from an IDE (such as eclipse) or from the command line (javac) then you will need to make sure to put the freepastry.params file located in jars/freepastry in your classpath, or use a different contstructor for the Environment.

My JVM won't exit, or How should I get rid of an environment?

The SelectorManager and Processor both have daemon threads that are used to handle communication and processing for the PastryNode. For the JVM to be able to exit, you will need to properly destroy the environment, by calling environment.destroy(). This will call store on the parameters, and destroy the selector/processor threads. Of course, you can also call System.exit(), but that is generally considered bad form.

This should be sufficient information for you to continue with the tutorial. You can learn more in Lesson 10 Advanced Environment.

Lesson 1

Minimal code to create/join a pastry ring.

Download the tutorial file DistTutorial.java.

In this lesson we are going to use Socket transport layer to communicate. The Socket transport layer is a package within the pastry source tree that is used to communicate between nodes using Internet Protocol (IP). It is located in rice.pastry.socket. Socket uses TCP for all messages except liveness which it uses UDP to accomplish. Before we dive into the code lets start with a short vocabulary lesson:

Terms:

Now that you have the proper vocabulary, here is the constructor for the DistTutorial. You can see most of the elements described in the vocabulary.

  /**
   * This constructor sets up a PastryNode.  It will bootstrap to an 
   * existing ring if it can find one at the specified location, otherwise
   * it will start a new ring.
   * 
   * @param bindport the local port to bind to 
   * @param bootaddress the IP:port of the node to boot from
   * @param env the environment for these nodes
   */
  public DistTutorial(int bindport, InetSocketAddress bootaddress, Environment env) throws Exception {
    
    // Generate the NodeIds Randomly
    NodeIdFactory nidFactory = new RandomNodeIdFactory(env);
    
    // construct the PastryNodeFactory, this is how we use rice.pastry.socket
    PastryNodeFactory factory = new SocketPastryNodeFactory(nidFactory, bindport, env);

    // This will return null if we there is no node at that location
    NodeHandle bootHandle = ((SocketPastryNodeFactory)factory).getNodeHandle(bootaddress);
      
    // construct a node, passing the null boothandle on the first loop will 
    // cause the node to start its own ring
    PastryNode node = factory.newNode(bootHandle);
      
    // the node may require sending several messages to fully boot into the ring
    while(!node.isReady()) {
      // delay so we don't busy-wait
      Thread.sleep(100);
    }
    
    System.out.println("Finished creating new node "+node);    
  }

Let's examine each line:

  1. The arguments that we start with are:
    1. int bindport — the local port to bind to.
    2. InetSocketAddress bootaddress — The address of our bootstrap node.
    3. Environment env — The environment. See lesson 0.b.

    public DistTutorial(int bindport, InetSocketAddress bootaddress, Environment env) throws Exception {
  2. We begin by constructing our NodeIdFactory, which we are going to need to give to our PastryNodeFactory.

    NodeIdFactory nidFactory = new RandomNodeIdFactory(env);
  3. Give the nidFactory to our SocketPastryNodeFactory. Additionally, we need to tell the SocketPastryNodeFactory what port to bind our pastry node to.

    PastryNodeFactory factory = new SocketPastryNodeFactory(nidFactory, bindport, env);
  4. Turn the bootaddress into a NodeHandle. Note that this call blocks, and can take several seconds to complete. It is opening a socket to the specified address. If there is an error, or no PastryNode found at the bootaddress, then getNodeHandle() returns null.

    NodeHandle bootHandle = ((SocketPastryNodeFactory)factory).getNodeHandle(bootaddress);
  5. Finally, create the PastryNode. If bootHandle is null then the factory will start a new ring.

    PastryNode node = factory.newNode(bootHandle);
  6. Even though we don't have an application to run yet, it is important that you are aware of the call to PastryNode.isReady(). This method returns false until the node is fully booted into the ring (which entails establishing his neighbor set i.e. the routing table and the leafset). This simple loop is a typical way to wait for the node to fully boot into the ring.
    while(!node.isReady()) {
      Thread.sleep(100);
    }
    

Before we can run our program lets take a quick look at the main() method. We need to get 1) the local port to bind to, 2) the IP address of the bootstrap node, and 3) the port of the bootstrap node.

  /**
   * Usage: 
   * java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort
   * example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001
   */
  public static void main(String[] args) throws Exception {
    // Loads pastry settings
    Environment env = new Environment();

    try {
      // the port to use locally
      int bindport = Integer.parseInt(args[0]);
      
      // build the bootaddress from the command line args
      InetAddress bootaddr = InetAddress.getByName(args[1]);
      int bootport = Integer.parseInt(args[2]);
      InetSocketAddress bootaddress = new InetSocketAddress(bootaddr,bootport);
  
      // launch our node!
      DistTutorial dt = new DistTutorial(bindport, bootaddress);
    } catch (Exception e) {
      // remind user how to use
      System.out.println("Usage:"); 
      System.out.println("java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort");
      System.out.println("example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001");
      throw e; 
    } 
  }

Let's examine each line:

  1. This line constructs the Environment. It has the side effect of starting a daemon thread.
        Environment env = new Environment();
    
  2. This line parses the first command line argument into an int.
        int bindport = Integer.parseInt(args[0]);
    
  3. These lines parse the IP and port and turn them into an InetSocketAddress.
        InetAddress bootaddr = InetAddress.getByName(args[1]);
        int bootport = Integer.parseInt(args[2]);
        InetSocketAddress bootaddress = new InetSocketAddress(bootaddr,bootport);
    
  4. Finally we execute our constructor.
        DistTutorial dt = new DistTutorial(bindport, bootaddress);
    
  5. We wrap the method with an indication of how to use the program in case the user inputs the wrong args.
        try {
    
          ...
          
        } catch (Exception e) {
          // remind user how to use
          System.out.println("Usage:"); 
          System.out.println("java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort");
          System.out.println("example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001");
          throw e; 
        } 
    

Congratulations! You have built code to launch/join a FreePastry ring!
Lesson 2 will aid you in running your code.


Lesson 2

Execute the code to launch your new ring.

This is a short lesson that shows you how to run DistTutorial.java which you created in Lesson 1.

Download the tutorial file DistTutorial.java.

After you compile the code, you can either run multiple nodes on 1 computer (but in separate processes) or if you have multiple computers, you can launch them on different machines as long as the computers can communicate with each other via IP. In other words, it won't work if the computers are behind different firewalls.

Congratulations! You have just launched your first FreePastry ring!
Lesson 3 will show you how to send and receive messages by creating a commonAPI application.


Lesson 3

Write a simple application using the commonAPI.

Download the tutorial files: DistTutorial.java (changed from Lesson 1!!!), MyApp.java, MyMsg.java into a directory called rice/tutorial/lesson3/.

This tutorial will show you how to create and run your first FreePastry application. You will be able to send/receive messages with this application.

Terms:

MyMsg

Let's start by taking a look at your message class. In FreePastry a Message is an Object that is Serializable. The transport layer serializes this object into bytes then sends it through the network. When the message is received, it is deserialized back into an Object and delivered to your application.

public class MyMsg implements Message {

This class implements rice.p2p.commonapi.Message. Message extends Serializable and has a single method: getPriority(). Let's take a look at that method now.

For now always return Message.LOW_PRIORITY for your messages. It is important to not set application message priority too high, or you may interfere with Pastry's overlay maintenance traffic that keeps the ring functioning properly.

  public int getPriority() {
    return Message.LOW_PRIORITY;
  }
The "payload" of your message is created by making member variables in your message. Here is the payload for MyMsg. An Id is the commonAPI version of a NodeId, it is also used as the "key" when routing.
  /**
   * Where the Message came from.
   */
  Id from;
  /**
   * Where the Message is going.
   */
  Id to;
We will create a toString() so that we can print out the message.
  public String toString() {
    return "MyMsg from "+from+" to "+to;
  }
Finally we have the constructor that loads the payload.
  public MyMsg(Id from, Id to) {
    this.from = from;
    this.to = to;
  }

MyApp

Now let's take a look at MyApp. MyApp is designed to log output whenever we send or receive a message.

The Endpoint is what we will call on to send messages.

  protected Endpoint endpoint;

The constructor generates an Endpoint from the node. The instance is designed to allow you to run the same app multiple times on the same node. The apps will not receive each other's messages. You will only be able to send messages to apps that generated endpoints with the same instance string. For most of your apps, you will only run one instance, so just make sure the instance is the same on all nodes.

  public MyApp(Node node) {
    // We are only going to use one instance of this application on each PastryNode
    this.endpoint = node.registerApplication(this, "myinstance");
  }

Sending a message:

In a Distributed Hash Table, or DHT, you typically want to route to the nearest node to the hash of an object that you are interested in. The commonAPI provides you with Endpoint.route() to accomplish this. This function sends a MyMsg to an id.

  /**
   * Called to route a message to the id
   */
  public void routeMyMsg(Id id) {
    System.out.println(this+" sending to "+id);    
    Message msg = new MyMsg(endpoint.getId(), id);
    endpoint.route(id, msg, null);
  }
Note that Endpoint.route() takes 3 arguments. They are:
  1. Id—the destination of the message (optional)
  2. Message—the message to send
  3. NodeHandle—a "hint": the node to route to first (optional)
Sometimes you need to send messages to a particular Node in the network, not just to the nearest key. For example, if you need to send messages that are large, they are going to take up a lot of bandwidth; it is more efficient to not route the messages through the overlay, but rather send directly to the node. You can use the Endpoint.route() method to accomplish this by passing a null argument as the Id and the target node's NodeHandle as the "hint" argument.
  /**
   * Called to directly send a message to the nh
   */
  public void routeMyMsgDirect(NodeHandle nh) {
    System.out.println(this+" sending direct to "+nh);
    Message msg = new MyMsg(endpoint.getId(), nh.getId());
    endpoint.route(null, msg, nh);
  }

Receiving a message:

Simply implement the deliver() method as is specified by the Application interface.

  /**
   * Called when we receive a message.
   */
  public void deliver(Id id, Message message) {
    System.out.println(this+" received "+message);
  }

For now you don't need to worry about the additional methods in the Application interface.

Congratulations, you have an application. Let's integrate it into DistTutorial.java

Here is the new code we will add to the bottom of the DistTutorial constructor:
    // construct a new MyApp
    MyApp app = new MyApp(node);
    
    // wait 10 seconds
    Thread.sleep(10000);
    
    // as long as we're not the first node
    if (bootHandle != null) {
      
      // route 10 messages
      for (int i = 0; i < 10; i++) {
        // pick a key at random
        Id randId = nidFactory.generateNodeId();
        
        // send to that key
        app.routeMyMsg(randId);
        
        // wait a sec
        Thread.sleep(1000);
      }

      // wait 10 seconds
      Thread.sleep(10000);
      
      // send directly to my leafset
      LeafSet leafSet = node.getLeafSet();
      
      // this is a typical loop to cover your leafset.  Note that if the leafset
      // overlaps, then duplicate nodes will be sent to twice
      for (int i=-leafSet.ccwSize(); i<=leafSet.cwSize(); i++) {
        if (i != 0) { // don't send to self
          // select the item
          NodeHandle nh = leafSet.get(i);
          
          // send the message directly to the node
          app.routeMyMsgDirect(nh);   
          
          // wait a sec
          Thread.sleep(1000);
        }
      }
    }

First we create the MyApp.

    // construct a new MyApp
    MyApp app = new MyApp(node);  
  

After waiting 10 seconds, let's send some messages, but only if I am not the first node. After all, if I'm the only node in the ring, it is hardly interesting to send messages to myself.

    if (bootHandle != null) {
Loop 10 times.
      for (int i = 0; i < 10; i++) {
Reuse the RandomNodeIdFactory to generate random keys to route to.
        Id randId = nidFactory.generateNodeId();
Route.
        app.routeMyMsg(randId);
Wait a second and repeat.
        Thread.sleep(1000);
      }

After waiting another 10 seconds, let's send some messages directly to nodes. This section is also going to show you how to access the leafset from the PastryNode. Note that this is a FreePastry specific call, not a commonAPI call.

Get the leafset from the PastryNode:

      LeafSet leafSet = node.getLeafSet();
Iterate over all of the nodes in the leafset.
      for (int i=-leafSet.ccwSize(); i<=leafSet.cwSize(); i++) {
Don't send to myself. The local node is node zero in the leafset.
        if (i != 0) { // don't send to self
Extract the nodehandle at that index.
          NodeHandle nh = leafSet.get(i);
Wait a second and repeat.
        Thread.sleep(1000);
      }
Send the message.
          app.routeMyMsgDirect(nh);   
      }
Now if you execute this code twice you should get something like:

(for Node1)

java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson3.DistTutorial 9001 10.9.8.7 9001
:1122933198281:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
:1122933198296:No bootstrap node provided, starting a new ring...
Finished creating new node SocketNodeHandle (<0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996])
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xA67C20..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xBF799E..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC4BEE7..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x86ACA9..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x9906E6..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x8F5015..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC20545..>
MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC20545..>

(for Node2)

java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson3.DistTutorial 9002 10.9.8.7 9001
Finished creating new node SocketNodeHandle (<0xDD90C6..>/FOO/10.9.8.7:9002 [5138450490561334965])
MyApp <0xDD90C6..> sending to <0x2E5C63..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x2E5C63..>
MyApp <0xDD90C6..> sending to <0x03045C..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x03045C..>
MyApp <0xDD90C6..> sending to <0xA67C20..>
MyApp <0xDD90C6..> sending to <0xF9C506..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0xF9C506..>
MyApp <0xDD90C6..> sending to <0xBF799E..>
MyApp <0xDD90C6..> sending to <0xC4BEE7..>
MyApp <0xDD90C6..> sending to <0x86ACA9..>
MyApp <0xDD90C6..> sending to <0x41F900..>
MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x41F900..>
MyApp <0xDD90C6..> sending to <0x9906E6..>
MyApp <0xDD90C6..> sending to <0x8F5015..>
MyApp <0xDD90C6..> sending direct to [SNH: <0xDD90C6..> -> <0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996]]
MyApp <0xDD90C6..> sending direct to [SNH: <0xDD90C6..> -> <0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996]]

Congratulations! You have just sent and received your first messages in FreePastry!