Java - Converting byte[] to hex string

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Prelates, Moderators General

Java - Converting byte[] to hex string

Postby wing » Sun Dec 30, 2007 3:58 am UTC

This works and everything, but I'm wondering if there's a better way to do it. I think I understand it, but it's not something that I'd instinctively produce and it strikes me as being not quite right.

Code: Select all
        public static String asHex(byte buf[])
        {
                StringBuffer strbuf = new StringBuffer(buf.length * 2);

                for(int i=0; i< buf.length; i++)
                {
                        if(((int) buf[i] & 0xff) < 0x10)
                                strbuf.append("0");
                        strbuf.append(Long.toString((int) buf[i] & 0xff, 16));
                }
                return strbuf.toString();
        }


Reason I ask is that this snippet actually came from the Sun Developers Network as part of an example that contained about half a million billion trillion mistakes/redundencies/unused variables/unused imports/bad form/general uselessness that I corrected in my implementation. A few of the things I corrected were symptoms of a poorly aging Hack who thinks it's really cool to use every operator in the language to do the maximum amount of stuff on one line rather than spreading it out across an if statement or something readable. In an example whose sole purpose is to be read.



Edit: Actually, nevermind. This looks like it checks out just fine. looking at DWIM implementations is always funny. (Why the hell is byte signed? I can't think of a single case where I'd want to use a byte to do anything other than store a number from 0 to 255)
I AM A SEXY, SHOELESS GOD OF WAR!
Akula wrote:Our team has turned into this hate-fueled juggernaut of profit. It's goddamn wonderful.
User avatar
wing
the /b/slayer
 
Posts: 1876
Joined: Tue May 29, 2007 5:56 am UTC

Re: Java - Converting byte[] to hex string

Postby Sc4Freak » Sun Dec 30, 2007 7:06 am UTC

I don't know about java, but this is how you'd do it in C++:

Code: Select all
using namespace std;
string asHex(vector<unsigned char> buf)
{
   ostringstream oss;
   for(vector<unsigned char>::iterator it = buf.begin(); it != buf.end; ++it)
      oss << std::hex << *it;
   return oss.str();
}
User avatar
Sc4Freak
 
Posts: 673
Joined: Thu Jul 12, 2007 4:50 am UTC
Location: Redmond, Washington

Re: Java - Converting byte[] to hex string

Postby Rysto » Sun Dec 30, 2007 6:42 pm UTC

That doesn't work. 0 needs to be encoded as "00".
Rysto
 
Posts: 1443
Joined: Wed Mar 21, 2007 4:07 am UTC

Re: Java - Converting byte[] to hex string

Postby Dropzone » Sun Dec 30, 2007 11:03 pm UTC

That code is, as you thought, not really the best way of doing it. For one thing, both of the casts to int are completely unnecessary (Eclipse pointed this out when I pasted the code in) - the type promotion rules mean that the bytes would get promoted to ints anyway. Also, the whole thing is neither particularly efficient nor particularly readable. I came up with three possible replacements... which one is the best depends on what exactly you're looking for.

Code: Select all
    public static String asHex(byte[] buf)
    {
        Formatter formatter = new Formatter();
        for (byte b : buf)
            formatter.format("%02x", b);
        return formatter.toString();
    }
I'd say that this is the most readable way of doing it. It is very slow (around 1/10 the speed of the original), but if speed wasn't important, this is probably what I'd use.

Code: Select all
    public static String asHex(byte[] buf)
    {
        String s = new BigInteger(1, buf).toString(16);
        return (s.length() % 2 == 0) ? s : "0" + s;
    }
This one's the shortest/laziest way I can think of. It's not exactly readable, and only about half as fast as the original, so I wouldn't actually recommend using it - it is quite an amusing trick though, I think.

Code: Select all
    private static final char[] HEX_CHARS = "0123456789abcdef".toCharArray();

    public static String asHex(byte[] buf)
    {
        char[] chars = new char[2 * buf.length];
        for (int i = 0; i < buf.length; ++i)
        {
            chars[2 * i] = HEX_CHARS[(buf[i] & 0xF0) >>> 4];
            chars[2 * i + 1] = HEX_CHARS[buf[i] & 0x0F];
        }
        return new String(chars);
    }
That's the fastest code I could come up with - it ran about 20 times faster than the original when I tested it. If speed was important, I'd use this rather than my first version.
User avatar
Dropzone
 
Posts: 405
Joined: Sun Dec 30, 2007 10:12 pm UTC
Location: North Lincs., UK

Re: Java - Converting byte[] to hex string

Postby Sc4Freak » Mon Dec 31, 2007 8:35 am UTC

Rysto wrote:That doesn't work. 0 needs to be encoded as "00".

Ah, right. Fixed:

Code: Select all
using namespace std;
string asHex(vector<unsigned char> buf)
{
   ostringstream oss;
   for(vector<unsigned char>::iterator it = buf.begin(); it != buf.end; ++it)
      oss << std::hex << std::setw(2) << std::setfill('0') << *it;
   return oss.str();
}
User avatar
Sc4Freak
 
Posts: 673
Joined: Thu Jul 12, 2007 4:50 am UTC
Location: Redmond, Washington

Re: Java - Converting byte[] to hex string

Postby hotaru » Thu Jan 03, 2008 2:54 am UTC

i don't do java or c++, but here's a way to do it in c that's probably a bit faster:
Code: Select all
char *as_hex(const size_t len, const uint8_t * const buf){
  // break loudly if char isn't 8 bits because we assume that two chars is 16 bits
  assert(CHAR_BIT == 8);
  static const uint16_t * const hex = (const uint16_t * const)(
    "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"
    "202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f"
    "404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f"
    "606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f"
    "808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9f"
    "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf"
    "c0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedf"
    "e0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff");
  uint16_t * ret = malloc(2 * len + 1);
  for(size_t i = 0; i < len; ++i) ret[i] = hex[buf[i]];
  ret[len]=0;
  return (char *)ret;
}


phpbb sucks for removing 1 space indentation, i had to use 2 spaces instead :x
Code: Select all
uint8_t f(uint8_t n)
{ if (!(
n&1)) return 2;
  if (
n==169) return 13; if (n==121||n==143) return 11;
  if (
n==77||n==91) return 7; if (n==3||n==5) return 0;
  
n=(n>>4)+(n&0xF); n+=n>>4n&=0xF;
  return (
n==3||n==6||n==9||n==12||n==15)?3:(n==5||n==10)?5:0; } 
User avatar
hotaru
 
Posts: 951
Joined: Fri Apr 13, 2007 6:54 pm UTC

Re: Java - Converting byte[] to hex string

Postby EvanED » Thu Jan 03, 2008 4:02 am UTC

hotaru wrote:i don't do java or c++, but here's a way to do it in c that's probably a bit faster:
Code: Select all
...
  // break loudly if char isn't 8 bits because we assume that two chars is 16 bits
  assert(CHAR_BIT == 8);

I would suggest a STATIC_ASSERT so that your code breaks at compile time rather than runtime, but that's just me.
EvanED
 
Posts: 4141
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI

Re: Java - Converting byte[] to hex string

Postby akashra » Thu Jan 03, 2008 4:12 am UTC

I agree. The way I'd do it is define an array with every byte value (so 256 entries) mapped to their string values, and look up that value, typecasting the byte[] data to a short to be used as an array index. Not the fastest due to the typecast, there would obviously be other ways to do it... but as a lookup is the quickest.

The slowest part will be appending the string. If you're really game, you can put the hex string straight into a byte[] array of the appropriate ascii/unicode characters, but that's a hell of a lot of effort - same principle here - and then convert the byte[] directly to a String.

Or you could just use JNI and use one of the existing really fast C methods to do so :)
( find / -name \*base\* -exec chown us : us { } \ ; )
akashra
 
Posts: 503
Joined: Tue Jan 01, 2008 6:54 am UTC
Location: Melbourne, AU

Re: Java - Converting byte[] to hex string

Postby Rysto » Thu Jan 03, 2008 4:23 am UTC

Why do a lookup into a 256-entry table? Dropzone's table-based method is much better IMO.
Rysto
 
Posts: 1443
Joined: Wed Mar 21, 2007 4:07 am UTC

Re: Java - Converting byte[] to hex string

Postby akashra » Thu Jan 03, 2008 4:32 am UTC

Because he asked how to do it in Java
( find / -name \*base\* -exec chown us : us { } \ ; )
akashra
 
Posts: 503
Joined: Tue Jan 01, 2008 6:54 am UTC
Location: Melbourne, AU

Re: Java - Converting byte[] to hex string

Postby Rysto » Thu Jan 03, 2008 4:34 am UTC

DropZone did do it in Java...
Rysto
 
Posts: 1443
Joined: Wed Mar 21, 2007 4:07 am UTC

Re: Java - Converting byte[] to hex string

Postby akashra » Thu Jan 03, 2008 4:57 am UTC

Sorry, I didn't see a table in any other post so thought you meant hotaru's solution, which was again slightly different.
You're right, what he's posted is pretty close to how I was explaining it. However, it's definately MUCH slower than how I suggested (at least 4 times from what I can tell).
( find / -name \*base\* -exec chown us : us { } \ ; )
akashra
 
Posts: 503
Joined: Tue Jan 01, 2008 6:54 am UTC
Location: Melbourne, AU

Re: Java - Converting byte[] to hex string

Postby Parsifal » Thu Feb 28, 2008 3:37 am UTC

I thought I would try using NIO to convert large byte arrays to hexadecimal strings using a 16-bit hex Charset as in http://www.exampledepot.com/egs/java.ni ... tChar.html - however,

I found that the faster method by Dropzone took on average about 18ms per 1,000,000 byte array, while the nio method took about 50ms (Although the Decoder has the capability to convert bytes to arbitrary numbers of characters each, and throws exceptions for unconvertible bytes). One thing that surprised me though - the naive method above which uses a java.util.Formatter took 3500ms per string!
Parsifal
 
Posts: 114
Joined: Thu Feb 28, 2008 1:35 am UTC

Re: Java - Converting byte[] to hex string

Postby Arancaytar » Fri Feb 29, 2008 1:19 pm UTC

Using a StringBuffer is better than appending to a String (the latter is inexcusably stupid, given that Java strings are immutable and appending to them requires copying the string every time). But I think that filling a char array has less overhead by using the fact that the length of the byte sequence is fixed and known. Java's char fills two bytes, I think, so this function allocates 4 times the input memory.

Code: Select all
static final char[] DIGITS = new char[] {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};

public String toHex(byte[] bytes) {
  char[] out = new char[bytes.length * 2]; // 2  hex characters per byte
  for (int i = 0; i < bytes.length; i++) {
    out[2*i] = DIGITS[bytes[i] < 0 ? 8 + (bytes[i] + 128) / 16 : bytes[i] / 16]; // append sign bit for negative bytes
    out[2*i + 1] = DIGITS[bytes[i] < 0 ? (bytes[i] + 128) % 16 : bytes[i] % 16];
  }
  return new String(out); // char sequence to string
}


Edit: Fixed a sign problem. IIRC, Java's byte goes from x00 = 0 .. x7f = 127, x80 = -128 .. xff = -1. Also, it appears that negative numbers produce a negative modulus.

Edit 2: Yes, I'm biased towards base types and against using Java's built-in converters. But as I'm pretty sure a byte[] -> hex conversion is somewhere in java.util, that seems kind of like the point of the exercise.
"You cannot dual-wield the sharks. One is enough." -Our DM.
Image
User avatar
Arancaytar
 
Posts: 1631
Joined: Thu Mar 15, 2007 12:54 am UTC
Location: 50.099432 degrees north, 8.572756 degrees east.

Re: Java - Converting byte[] to hex string

Postby Berengal » Fri Feb 29, 2008 1:59 pm UTC

Code: Select all
String output = "";
for(int i = 0; i < array.length/4; i++){
  int ii = i * 4;
  int inty = array[ii] + array[ii+1]*0xff + array[ii+2]*0xffff + array[ii+3]*0xffffff;
  output = Integer.toHexString(inty) + output;
}


Various methods for producing the string not included. Also, not too sure about how the speed of the whole thing, but I generally don't worry too much about that (unless the system becomes genuinely slow). You could also just do Integer.toHexString(byte). Not too sure about the difference in speed, actually...
It is practically impossible to teach good programming to students who are motivated by money: As potential programmers they are mentally mutilated beyond hope of regeneration.
User avatar
Berengal
Superabacus Mystic of the First Rank
 
Posts: 2707
Joined: Thu May 24, 2007 5:51 am UTC
Location: Bergen, Norway

Re: Java - Converting byte[] to hex string

Postby zahlman » Sat Mar 01, 2008 10:31 pm UTC

Dropzone wrote:
Code: Select all
    private static final char[] HEX_CHARS = "0123456789abcdef".toCharArray();

    public static String asHex(byte[] buf)
    {
        char[] chars = new char[2 * buf.length];
        for (int i = 0; i < buf.length; ++i)
        {
            chars[2 * i] = HEX_CHARS[(buf[i] & 0xF0) >>> 4];
            chars[2 * i + 1] = HEX_CHARS[buf[i] & 0x0F];
        }
        return new String(chars);
    }
That's the fastest code I could come up with - it ran about 20 times faster than the original when I tested it. If speed was important, I'd use this rather than my first version.


I would use this kind of thing because it's unlikely to need maintenance and the name is as descriptive as needed. (I'm also very accustomed to writing lower-level stuff like this in higher-level languages ;) )

Just for fun, a version with a precomputed table of pairs of chars:

Code: Select all
private static final char[] byteToHexPair;

static {
  // Optimizing this part too, just for the heck of it. Sure it only runs once ;/
  // javac does surprisingly little optimization by default, expecting the JIT to take care of
  // things. My background is in J2ME work on very restricted phones where you don't have that
  // luxury. I'll spare myself the pain of doing all the internal work with byte[] arrays.
  char[] hexDigits = "0123456789abcdef".toCharArray();
  byteToHexPair = new char[16 * 16 * 2];
  for (int i = 0, o = 0; i < 256; ++i) {
    byteToHexPair[o++] = hexDigits[i >>> 4];
    byteToHexPair[o++] = hexDigits[i & 15];
  }
}

public static String asHex(byte[] buf) {
  int size = buf.length;
  char[] chars = new char[2 * size];
  for (int i = 0, o = 0; i < size; ++i) {
    int index = buf[i];
    chars[o++] = byteToHexPair[index++];
    chars[o++] = byteToHexPair[index];
  }
  return new String(chars);
}
Belial wrote:I once had a series of undocumented and nonstandardized subjective experiences that indicated that anecdotal data is biased and unreliable.
zahlman
 
Posts: 638
Joined: Wed Jan 30, 2008 5:15 pm UTC


Return to Coding

Who is online

Users browsing this forum: No registered users and 6 guests