Java - Converting byte[] to hex string

[wing]

This works and everything, but I'm wondering if there's a better way to do it. I think I understand it, but it's not something that I'd instinctively produce and it strikes me as being not quite right.

Code: Select all

        public static String asHex(byte buf[])
        {
                StringBuffer strbuf = new StringBuffer(buf.length * 2);

                for(int i=0; i< buf.length; i++)
                {
                        if(((int) buf[i] & 0xff) < 0x10)
                                strbuf.append("0");
                        strbuf.append(Long.toString((int) buf[i] & 0xff, 16));
                }
                return strbuf.toString();
        }


Reason I ask is that this snippet actually came from the Sun Developers Network as part of an example that contained about half a million billion trillion mistakes/redundencies/unused variables/unused imports/bad form/general uselessness that I corrected in my implementation. A few of the things I corrected were symptoms of a poorly aging Hack who thinks it's really cool to use every operator in the language to do the maximum amount of stuff on one line rather than spreading it out across an if statement or something readable. In an example whose sole purpose is to be read.



Edit: Actually, nevermind. This looks like it checks out just fine. looking at DWIM implementations is always funny. (Why the hell is byte signed? I can't think of a single case where I'd want to use a byte to do anything other than store a number from 0 to 255)

[Sc4Freak]

I don't know about java, but this is how you'd do it in C++:

Code: Select all

using namespace std;
string asHex(vector<unsigned char> buf)
{
   ostringstream oss;
   for(vector<unsigned char>::iterator it = buf.begin(); it != buf.end; ++it)
      oss << std::hex << *it;
   return oss.str();
}

[Rysto]

That doesn't work. 0 needs to be encoded as "00".

[Dropzone]

That code is, as you thought, not really the best way of doing it. For one thing, both of the casts to int are completely unnecessary (Eclipse pointed this out when I pasted the code in) - the type promotion rules mean that the bytes would get promoted to ints anyway. Also, the whole thing is neither particularly efficient nor particularly readable. I came up with three possible replacements... which one is the best depends on what exactly you're looking for.

Code: Select all

    public static String asHex(byte[] buf)
    {
        Formatter formatter = new Formatter();
        for (byte b : buf)
            formatter.format("%02x", b);
        return formatter.toString();
    }

I'd say that this is the most readable way of doing it. It is very slow (around ¹/₁₀ the speed of the original), but if speed wasn't important, this is probably what I'd use.

Code: Select all

    public static String asHex(byte[] buf)
    {
        String s = new BigInteger(1, buf).toString(16);
        return (s.length() % 2 == 0) ? s : "0" + s;
    }

This one's the shortest/laziest way I can think of. It's not exactly readable, and only about half as fast as the original, so I wouldn't actually recommend using it - it is quite an amusing trick though, I think.

Code: Select all

    private static final char[] HEX_CHARS = "0123456789abcdef".toCharArray();

    public static String asHex(byte[] buf)
    {
        char[] chars = new char[2 * buf.length];
        for (int i = 0; i < buf.length; ++i)
        {
            chars[2 * i] = HEX_CHARS[(buf[i] & 0xF0) >>> 4];
            chars[2 * i + 1] = HEX_CHARS[buf[i] & 0x0F];
        }
        return new String(chars);
    }

That's the fastest code I could come up with - it ran about 20 times faster than the original when I tested it. If speed was important, I'd use this rather than my first version.

[Sc4Freak]

That doesn't work. 0 needs to be encoded as "00".

Ah, right. Fixed:

Code: Select all

using namespace std;
string asHex(vector<unsigned char> buf)
{
   ostringstream oss;
   for(vector<unsigned char>::iterator it = buf.begin(); it != buf.end; ++it)
      oss << std::hex << std::setw(2) << std::setfill('0') << *it;
   return oss.str();
}

[hotaru]

i don't do java or c++, but here's a way to do it in c that's probably a bit faster:

Code: Select all

char *as_hex(const size_t len, const uint8_t * const buf){
  // break loudly if char isn't 8 bits because we assume that two chars is 16 bits
  assert(CHAR_BIT == 8);
  static const uint16_t * const hex = (const uint16_t * const)(
    "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"
    "202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f"
    "404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f"
    "606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f"
    "808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9f"
    "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf"
    "c0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedf"
    "e0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff");
  uint16_t * ret = malloc(2 * len + 1);
  for(size_t i = 0; i < len; ++i) ret[i] = hex[buf[i]];
  ret[len]=0;
  return (char *)ret;
}

phpbb sucks for removing 1 space indentation, i had to use 2 spaces instead

[EvanED]

i don't do java or c++, but here's a way to do it in c that's probably a bit faster:

Code: Select all

...
  // break loudly if char isn't 8 bits because we assume that two chars is 16 bits
  assert(CHAR_BIT == 8);

I would suggest a STATIC_ASSERT so that your code breaks at compile time rather than runtime, but that's just me.

[akashra]

I agree. The way I'd do it is define an array with every byte value (so 256 entries) mapped to their string values, and look up that value, typecasting the byte[] data to a short to be used as an array index. Not the fastest due to the typecast, there would obviously be other ways to do it... but as a lookup is the quickest.

The slowest part will be appending the string. If you're really game, you can put the hex string straight into a byte[] array of the appropriate ascii/unicode characters, but that's a hell of a lot of effort - same principle here - and then convert the byte[] directly to a String.

Or you could just use JNI and use one of the existing really fast C methods to do so

[Rysto]

Why do a lookup into a 256-entry table? Dropzone's table-based method is much better IMO.

[akashra]

Because he asked how to do it in Java

[Rysto]

DropZone did do it in Java...

[akashra]

Sorry, I didn't see a table in any other post so thought you meant hotaru's solution, which was again slightly different.
You're right, what he's posted is pretty close to how I was explaining it. However, it's definately MUCH slower than how I suggested (at least 4 times from what I can tell).

[Parsifal]

I thought I would try using NIO to convert large byte arrays to hexadecimal strings using a 16-bit hex Charset as in http://www.exampledepot.com/egs/java.ni ... tChar.html - however,

I found that the faster method by Dropzone took on average about 18ms per 1,000,000 byte array, while the nio method took about 50ms (Although the Decoder has the capability to convert bytes to arbitrary numbers of characters each, and throws exceptions for unconvertible bytes). One thing that surprised me though - the naive method above which uses a java.util.Formatter took 3500ms per string!

[Arancaytar]

Using a StringBuffer is better than appending to a String (the latter is inexcusably stupid, given that Java strings are immutable and appending to them requires copying the string every time). But I think that filling a char array has less overhead by using the fact that the length of the byte sequence is fixed and known. Java's char fills two bytes, I think, so this function allocates 4 times the input memory.

Code: Select all

static final char[] DIGITS = new char[] {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};

public String toHex(byte[] bytes) {
  char[] out = new char[bytes.length * 2]; // 2  hex characters per byte
  for (int i = 0; i < bytes.length; i++) {
    out[2*i] = DIGITS[bytes[i] < 0 ? 8 + (bytes[i] + 128) / 16 : bytes[i] / 16]; // append sign bit for negative bytes
    out[2*i + 1] = DIGITS[bytes[i] < 0 ? (bytes[i] + 128) % 16 : bytes[i] % 16];
  }
  return new String(out); // char sequence to string
}


Edit: Fixed a sign problem. IIRC, Java's byte goes from x00 = 0 .. x7f = 127, x80 = -128 .. xff = -1. Also, it appears that negative numbers produce a negative modulus.

Edit 2: Yes, I'm biased towards base types and against using Java's built-in converters. But as I'm pretty sure a byte[] -> hex conversion is somewhere in java.util, that seems kind of like the point of the exercise.

[Berengal]

Code: Select all

String output = "";
for(int i = 0; i < array.length/4; i++){
  int ii = i * 4;
  int inty = array[ii] + array[ii+1]*0xff + array[ii+2]*0xffff + array[ii+3]*0xffffff;
  output = Integer.toHexString(inty) + output;
}

Various methods for producing the string not included. Also, not too sure about how the speed of the whole thing, but I generally don't worry too much about that (unless the system becomes genuinely slow). You could also just do Integer.toHexString(byte). Not too sure about the difference in speed, actually...

[zahlman]

Code: Select all

    private static final char[] HEX_CHARS = "0123456789abcdef".toCharArray();

    public static String asHex(byte[] buf)
    {
        char[] chars = new char[2 * buf.length];
        for (int i = 0; i < buf.length; ++i)
        {
            chars[2 * i] = HEX_CHARS[(buf[i] & 0xF0) >>> 4];
            chars[2 * i + 1] = HEX_CHARS[buf[i] & 0x0F];
        }
        return new String(chars);
    }

That's the fastest code I could come up with - it ran about 20 times faster than the original when I tested it. If speed was important, I'd use this rather than my first version.

I would use this kind of thing because it's unlikely to need maintenance and the name is as descriptive as needed. (I'm also very accustomed to writing lower-level stuff like this in higher-level languages )

Just for fun, a version with a precomputed table of pairs of chars:

Code: Select all

private static final char[] byteToHexPair;

static {
  // Optimizing this part too, just for the heck of it. Sure it only runs once ;/
  // javac does surprisingly little optimization by default, expecting the JIT to take care of
  // things. My background is in J2ME work on very restricted phones where you don't have that
  // luxury. I'll spare myself the pain of doing all the internal work with byte[] arrays.
  char[] hexDigits = "0123456789abcdef".toCharArray();
  byteToHexPair = new char[16 * 16 * 2];
  for (int i = 0, o = 0; i < 256; ++i) {
    byteToHexPair[o++] = hexDigits[i >>> 4];
    byteToHexPair[o++] = hexDigits[i & 15];
  }
}

public static String asHex(byte[] buf) {
  int size = buf.length;
  char[] chars = new char[2 * size];
  for (int i = 0, o = 0; i < size; ++i) {
    int index = buf[i];
    chars[o++] = byteToHexPair[index++];
    chars[o++] = byteToHexPair[index];
  }
  return new String(chars);
}