Should matrices be row-major or column-major?

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

User avatar
Qaanol
The Cheshirest Catamount
Posts: 3035
Joined: Sat May 09, 2009 11:55 pm UTC

Should matrices be row-major or column-major?

Postby Qaanol » Fri Sep 02, 2016 2:14 am UTC

Should 2-dimensional matrices be stored in row-major or column-major order?

There does not seem to be any consensus among languages and libraries:

Wikipedia wrote:Row-major order is used in C/C++/Objective-C (for C-style arrays), Mathematica, PL/I, Pascal, Speakeasy, SAS, and C#/CLI/.Net.

Column-major order is used in Fortran, OpenGL and OpenGL ES, MATLAB, GNU Octave, S-Plus, R, Julia, Rasdaman, Scilab.

Row-major order is the default in NumPy (for Python).


There is also the BLAS approach where operations take flags to indicate which matrices are transposed.
wee free kings

Tub
Posts: 311
Joined: Wed Jul 27, 2011 3:13 pm UTC

Re: Should matrices be row-major or column-major?

Postby Tub » Fri Sep 02, 2016 10:42 am UTC

That depends. What data are you putting in there?

If a row of data is meaningful on its own (like in a database table), use row-major. If it's just a matrix that's only meaningful as a whole, it doesn't matter much.

If you're really worried about doing it wrong, consider fitting the memory model to your preferred iteration order. Traversing memory linearly increases cache locality, which can have an effect on your performance if your matrices are large. Also take a look at SIMD instructions, they only work on values in adjacent memory locations.

Bottom line, for matrix multiplication, you obviously store one of them in row-major and the other in column-major for maximum performance. That totally does not complicate things and will certainly not lead to bugs or other overhead that outweighs the gains from faster multiplication.

User avatar
Flumble
Yes Man
Posts: 1944
Joined: Sun Aug 05, 2012 9:35 pm UTC

Re: Should matrices be row-major or column-major?

Postby Flumble » Fri Sep 02, 2016 11:57 pm UTC

I don't even know which is which, but since OpenGL wants column-major and graphics (or physics or whatever's up to 4 dimensions) are the only valid use of matrices, column-major is the right choice. No matter how counter-intuitive.

Tub wrote:Bottom line, for matrix multiplication, you obviously store one of them in row-major and the other in column-major for maximum performance. That totally does not complicate things and will certainly not lead to bugs or other overhead that outweighs the gains from faster multiplication.

Totally agree. Though I hope that at the very least modern GPUs have a special (single cycle!) instruction for 4x4 matrix multiplication.

Is there any reason why this thread is not in Religious Wars?

User avatar
pogrmman
Posts: 324
Joined: Wed Jun 29, 2016 10:53 pm UTC
Location: Probably outside

Re: Should matrices be row-major or column-major?

Postby pogrmman » Sun Sep 04, 2016 1:26 pm UTC

Flumble wrote:I don't even know which is which, but since OpenGL wants column-major and graphics (or physics or whatever's up to 4 dimensions) are the only valid use of matrices, column-major is the right choice. No matter how counter-intuitive.


Graphics and physics certainly aren't the only valid uses of matrices. For instance, try implementing a neural net without matrices -- it's possible, but it's a total waste of memory and code for the vast majority of neural net architectures.

Matrices are great if you've got to store and process some serif values, which can come up in loads of different areas.

EmelinaVollmering
Posts: 11
Joined: Tue Oct 18, 2016 10:24 am UTC

Re: Should matrices be row-major or column-major?

Postby EmelinaVollmering » Tue Oct 18, 2016 10:55 am UTC

I does not matter. Most of the times, teacher use the row-major. And it is being followed from one generation to another. You can also use the column-major instead and make the analogues changing.

In my case, I will see, which way lower number of operations are performed. I will use row-major or column-major in that direction.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 10 guests