Socket Direct Protocol

SDP is a wire protocol to support stream sockets networking over IB. It utilizes features such as RDMA for high performance data transfers and provides higher performance and lower latency compared to IP encapsulation over IB (IPoIB). A long standing request has been to add implementation-specific support to the JDK to make it easy to use SDP in Java Language applications. As SDP provides a streams sockets interfaces it can be made transparent to applications using java.net.Socket/ServerSocket and of course the stream-oriented SelectableChannels and AsynchronousChannels in the channels package.
The simple solution in jdk7 uses a configuration file with rules to indicate the endpoints that are connected to the IB fabric for when SDP should be used. When a java.net.Socket or java.nio.channels.SocketChannel binds or connects to an address that is an endpoint on the IB fabric then the socket is converted to use the SDP protocol. The solution can be thought of a kind of built-in interposer library and using an interposer library is an alternative way of doing this.

There are two types of rules, as follows:


A "bind" rule indicates that the SDP protocol transport should be used when a TCP socket binds to an address and port that match the rule.
A "connect" rule indicates that the SDP protocol transport should be used when an unbound TCP socket attempts to connect to an address and port that match the rule.
A rule has the following form:


("bind"|"connect")1*LWSP-char(hostname|ipaddress)["/"prefix])1*LWSP-char("*"|port)["-"("*"|port)]



The first keyword indicates whether the rule is a bind or a connect rule. The next token specifies either a host name or a literal IP address. When you specify a literal IP address, you can also specify a prefix, which indicates an IP address range. The third and final token is a port number or a range of port numbers.


Consider the following notation in this sample configuration file:


# Use SDP when binding to 192.168.1.1
bind 192.168.1.1 *


# Use SDP when connecting to all application services on 192.168.1.*
connect 192.168.1.0/24     1024-*



The first rule in the sample file specifies that SDP is used for any port (*) on the local IP address 192.168.1.1. You would add a bind rule for each local address assigned to an InfiniBand adaptor. (An InfiniBand adaptor is the equivalent of a network interface card (NIC) for InfiniBand.) If you had several IB adaptors, you would use a bind rule for each address that is assigned to those adaptors.


The second rule in the sample file specifies that whenever connecting to 192.168.1.* and the target port is 1024 or greater, SDP is used. The prefix on the IP address /24 indicates that the first 24 bits of the 32-bit IP address should match the specified address. Each portion of the IP address uses 8 bits, so 24 bits indicates that the IP address should match 192.168.1 and the final byte can be any value. The -* notation on the port token specifies "and above." A range of ports, such as 1024—2056, would also be valid and would include the end points of the specified range.


The final rules in the sample file specify a host name (hpccluster), first with the port assigned to an http server (80) and then with the port assigned to a MySQL database (3306). Unlike a literal IP address, a host name can translate into multiple addresses. When you specify a host name, it matches all addresses that the host name is registered to in the name service.

Once the configuration file is created we simply specify it when running the application, 
     eg:$ java -Dcom.sun.sdp.conf=sdp.conf -Djava.net.preferIPv4Stack=true MyApplication
Note that this example also sets the java.net.preferIPv4Stack property. The Java Runtime always uses IPv6 sockets if IPv6 is enabled. The Solaris implementation of SDP supports IPv6 addresses but currently doesn't support IPv4-mapped IPv6 addresses (::ffff:192.168.1.1 for example). For now this means setting the property so that all sockets are IPv4 sockets.



To test that SDP is enabled, use the sdpadm(1M) command:
% /usr/sbin/sdpadm status
SDP is Enabled
Other commands you might find useful are ib(7D), ibd(7D), and sdp(7D).


You can use the grep command to search the /etc/path_to_inst file for the string "ibd" to view a list of IB adaptors that are supported on your network.