Inserting data with Thrift and Cassandra 0.7

A lot has changed from Cassandra 0.6 to 0.7, and sometimes it is hard to find examples of how things work. I'll be posting how to's on some of the most usual operations you might want to perform when using Cassandra, written in Java.

First of you have to establish a connection to the server:

TFramedTransport transport = new TFramedTransport(new TSocket("localhost", 9160));
Cassandra.Client client = new Cassandra.Client(new TBinaryProtocol(transport));
transport.open();

Here I'm using localhost and the default port for Cassandra, but this can be configured.

One difference to the previous versions of Cassandra is that the connection can be bound to a keyspace, and can be set as so:

client.set_keyspace("Keyspace");

With the connection established you need only the data to insert, now. This data is passed to the server in the form of mutations (org.apache.cassandra.thrift.Mutation).

In this example, I'll be adding a column to a row in a column family in the predefined keyspace.


List<Mutation> insertion_list = new ArrayList<Mutation>();

Column col_to_add = new Column(ByteBuffer.wrap(("name").getBytes("UTF8")), ByteBuffer.wrap(("value").getBytes("UTF8")),System.currentTimeMillis());

Mutation mut = new Mutation();
mut.setColumn_or_supercolumn(new ColumnOrSuperColumn().setColumn(col_to_add));
insertion_list.add(mut);

Map<String,List<Mutation>> columnFamilyValues = new HashMap<String,List<Mutation>>();
columnFamilyValues.put("columnFamily",insertion_list);

Map<ByteBuffer,<String,List<Mutation>>> rowDefinition = new HashMap<ByteBuffer,<String,List<Mutation>>>();
rowDefinition.put(ByteBuffer.wrap(("key").getBytes("UTF8")),columnFamilyValues);

client.batch_mutate(rowDefinition,ConsistencyLevel.ONE);


The code is pretty much self explaining, apart from some values that can be reconfigured at will, as the encoding of the strings (I've used UTF8), and the consistency level of the insertion (I've used ONE).

In the case of the consistency levels you should check out Cassandra's wiki, to better understand it's usage.

To close the connection to the server it as easy as,

transport.close();

Hope you find this useful. Next I'll give an example of how to get data from the server, as soon as I have some time. ;)

Java static blocks

During the course of the year I found out about a very interesting thing in Java, the static blocks. Java has a special word for variables that are initialized once per class, instead of once per instance of the class. That word is static.

static int class_variable = 0;

This way, all the instances of the class share this variable (this also means you have to be careful about concurrency control). Now, what if you want to do this with a Collection, instead of a primitive type?

That's where static blocks come in handy. Imagine we have this:

static HashMap<Integer,String> sharedHash = new HashMap<Integer,String>();

What we want now is to populate the HashMap, either from hardcoded values or a function that does the job. The solution is simple, a static block!

static
{
    sharedHash.put(1,"one");
    this.populateHash(sharedHash);
}

This populates the HashMap only once, and only when the first instance of the class is created, saving a lot of unnecessary processing and memory usage.

DER Signing using Java and Bouncy Castle

Following my last post about DER coding in Java, I will now explain how to sign an object, using java.security.PrivateKey, java.security.cert.X509Certificate and the TSL standard as pointed in that post.

The TSL signature is an enveloped signature, that can be described by the following picture:

First let's assume that we have some previously initialized variables:

• tslDER - Unsigned TSL object (instance of DEREncodable)
• privateKey - Private key being used to sign
• cert - PKI Certificate of the signer (this one is optional, but recommended)

Now let's define the algorithms used for signing and message digesting:

AlgorithmIdentifier sigAlg = new AlgorithmIdentifier(ObjectIdentifiers.getAlgorithmOID(sigAlgorithm));
ASN1Set sigAlgs = new DERSet(sigAlg);
AlgorithmIdentifier digestAlg = new AlgorithmIdentifier(ObjectIdentifiers.getAlgorithmOID(digestAlgorithm));
ASN1Set digestAlgs = new DERSet(digestAlg);

The getAlgorithmOID is a method that given a String returns the matching Object Identifier or throws an Exception.

After that, let's define the identification of the signer:

SignerIdentifier sid = null;
String[] names = cert.getIssuerDN().getName().split(",");

ASN1EncodableVector vec = new ASN1EncodableVector();
for (String name : names)
{
    ASN1EncodableVector nameVec = new ASN1EncodableVector();
    String[] vals = name.split("=");
    nameVec.add((DEREncodable) X509Name.DefaultLookUp.get(vals[0].trim().toLowerCase()));
    nameVec.add(new DERPrintableString(vals[1]));
    vec.add(new DERSequence(nameVec));
}

ASN1Set attr = new DERSet(vec);
X509Name issuer = new X509Name(new DERSequence(attr));
sid = new SignerIdentifier(new IssuerAndSerialNumber(issuer, cert.getSerialNumber()));

Now the encapsulated content info:

ContentInfo encapContent = new ContentInfo(ObjectIdentifiers.id_eContentType_signedTSL, tslDER);

The next step is building the byte array to be signed:


MessageDigest md = MessageDigest.getInstance(digestAlgorithm);
md.reset();
md.update(tslDER.getDERObject().getEncoded());

ASN1EncodableVector attrs = new ASN1EncodableVector();

ASN1EncodableVector contentTypeAttr = new ASN1EncodableVector();
contentTypeAttr.add(ObjectIdentifiers.id_contentType);
contentTypeAttr.add(ObjectIdentifiers.id_eContentType_signedTSL);
attrs.add(new DERSequence(contentTypeAttr));

ASN1EncodableVector messageDigestAttr = new ASN1EncodableVector();
messageDigestAttr.add(ObjectIdentifiers.id_messageDigest);
messageDigestAttr.add(new DEROctetString(md.digest()));
attrs.add(new DERSequence(messageDigestAttr));

ASN1Set signedAttr = new DERSet(attrs);

md.reset();
md.update(tslDER.getDERObject().getDEREncoded());
md.update(signedAttr.getDEREncoded());
byte[] data = md.digest();


And to finalize the signature:


Signature sig = Signature.getInstance(sigAlgorithm);
sig.initSign(privateKey);
sig.update(data);
byte[] signed = sig.sign();

SignerInfo signerInfo = new SignerInfo(sid, digestAlg, signedAttr, sigAlg, new DEROctetString(signed), null);
ASN1Set signerInfos = new DERSet(signerInfo);

ASN1Set certs = new DERSet(new DEROctetString(cert.getEncoded()));

SignedData signedData = new SignedData(digestAlgs, encapContent, certs, null, signerInfos);
ContentInfo content = new ContentInfo(CMSObjectIdentifiers.signedData, signedData);


DER using Java and Bouncy Castle

In the last few months I have been working with Distinguished Encoding Rules (DER) of ASN.1 a standard placed by X.509, used in cryptography. I needed to code this in Java, for which I chose to use the Bouncy Castle ASN.1 library.

DER has some primitives that let's you define the various structures and types needed for your endeavors. Here are some:

• Sequence
• Choice
• IA5String
• PrintableString
• Integer
• Object Identifier
• GeneralizedTime
• Boolean
• OctetString

Any of these types can be marked as OPTIONAL and have a tag, in the form [n].

As an example I shall use the Trust-service Status List ETSI TS 102 231 V3.1.2 standard specification.

A sequence:

LangPointer ::= SEQUENCE {
  languageTag LanguageTag,
  uRI         NonEmptyURI
  } 

Encoding:

ASN1EncodableVector vec = new ASN1EncodableVector();
vec.add(LanguageTag);
vec.add(NonEmptyURI);
DERSequence seq = new DERSequence(vec);

Decoding:


DEREncodable obj = ...;
DERSequence tslscheme = (DERSequence) obj;
Enumeration e = tslscheme.getObjects();
LanguageTag lt = (LanguageTag) e.nextElement();

A choice:

TSLpolicy ::= CHOICE {
 pointer [0] MultiLangPointer,
 text [1] MultiLangString
}

Encoding (using a sequence):


if(multiLangPointer != null)
    vec.add(new DERTaggedObject(0, MultiLangPointer));
else if(multiLangString !=null)
    vec.add(new DERTaggedObject(1, MultiLangString));


Decoding:


while(e.hasMoreElements())
{
    DERTaggedObject tagged = (DERTaggedObject) e.nextElement();
    switch (tagged.getTagNo())
    {
        case 0: multiLangPointer = tagged.getObject(); break;
        case 1: multiLangString = tagged.getObject(); break;
    }
}


The primitive types are all encoded and decoded in a similar way, the only thing changing being the name, therefore I will only give one example:

NonEmptyURI ::= IA5String (SIZE (1..MAX))

Encoding:

DERIA5String neuri = new DERIA5String(String);

Decoding (in a sequence):

DERIA5String neuri = (DERIA5String) e.nextElement();
String str = type.getString();

A little bit trickier is casting a DERGeneralizedTime to a GregorianCalendar, but here's how you do it:

DERGeneralizedTime issueTime = (DERGeneralizedTime) e.nextElement();
SimpleDateFormat f = new SimpleDateFormat("yyyyMMddkkmmss");
GregorianCalendar greg = new GregorianCalendar();
greg.setTime(f.parse(issueTime.getTimeString()));


The date format can be changed as you see fit, to do what you need.

The last thing are the OPTIONALS, that are very much like a CHOICE, except that there can be more than one at the same time. Even though, the encoding and decoding are done in the same way, using DERTaggedObjects.

To write and read from DER encoded files, you can use ASN1Streams, initialized like this:

ASN1InputStream ain = new ASN1InputStream(new DataInputStream(new BufferedInputStream(new FileInputStream(new File(file)))));