Coding: Fleeting Thoughts

[elminster]

Amongst other things, we recently switched from using svn to git... my mind was blown a little after learning about it and trying it out. Like instant solutions for a whole ranges of problems with svn.

[Aaeriele]

Amongst other things, we recently switched from using svn to git... my mind was blown a little after learning about it and trying it out. Like instant solutions for a whole ranges of problems with svn.

Yes, Git is quite awesome for most things. There are a couple of corner cases where Subversion performs better, but they're few and far between.

[Thesh]

Subversion works for things like binary files or SSIS packages, where you can't have two people working on them on the same time (you must lock the file to edit it).

My question is why people compare SVN/CVS and Git/Mercurial as the differences between centralized vs distributed version control (only the "many backups" thing is an advantage of distributed, but even then, not really). Git/Mercurial don't work well because they are distributed, they work well because they don't force you to commit changes to a central repository if you want to create what is essentially a checkpoint, and they work because they handle commits and merges in terms of changesets instead of revisions, which can significantly cut back on conflicts when merging branches. If there was a centralized repository that allowed you to create checkpoints that other devs don't get (while still being saved on a centralized, backed up server) and was based on changesets, I think it would be more versatile.

[Jplus]

Local commits also save time.

[EvanED]

If there was a centralized repository that allowed you to create checkpoints that other devs don't get (while still being saved on a centralized, backed up server) and was based on changesets, I think it would be more versatile.

Until you want to work on the bus or plane.

Though I do agree that, for a centralized-organization-type project where the version control doesn't present any new access issues, most of the benefits of Git over Subversion are "artifical" in the sense that a centralized VCS could provide the appropriate features (e.g. something like git add --patch/--interactive), but they just don't for whatever reason.

[Aaeriele]

1. Local commits are faster.
2. Local clone means you can work from anywhere.
3. Local clone means you can keep work private until you want to publish it.
4. Local branching is faster.
5. Local history access is faster.

Those are the main reasons why distributed is inherently nice. Basically, it mostly comes down to speed - a centralized-server-based VCS is never going to beat a distributed VCS for pure speed, due to having local access to history.

[EvanED]

1. Local commits are faster.
2. Local clone means you can work from anywhere.
3. Local clone means you can keep work private until you want to publish it.
4. Local branching is faster.
5. Local history access is faster.

Those are the main reasons why distributed is inherently nice. Basically, it mostly comes down to speed - a centralized-server-based VCS is never going to beat a distributed VCS for pure speed, due to having local access to history.

Re. speed, I'd say that depends on your setup. I'd dispute the benefits for a company type situation where most work is getting done at the office. With a decent network, even centralized operations are pretty fast in that scenario. I'd bet that I've lost more time because I locally committed and forgot to push than I've saved because local operations are theoretically faster. (I know I'm not the only one to have that problem either. I fixed it by getting into the habit of almost always pushing right after committing, which negates your #1.)

Further, I'd argue #3 isn't a clear benefit. The key thing that VCS needs to support along that line for a business scenario is lightweight branches. (Maybe some way of keeping track of a tree of branches, rather than just having everything be a single name. That way you could do something like branch evaned/feature_a/toying_with_subfeature_b. Interestingly enough that's one of a couple benefits that Subversion has in the branch department, though they're not enough by a longshot to override the fact that they are otherwise obnoxious. ) Keeping it private seems both unnecessary and, in some cases, the opposite of what you want if you're running a dev company.

I'm sure that DVCSs are a godsend for open source projects and other projects which are actually distributed, but I stand by what I said before: for projects which are centralized by their nature, DVCSs buy you two things: (1) disconnected operation (which can very important, and is pretty distributed-by-nature, though you could also imagine a hybrid system) and (2) a ton of nice things that are present in DVCSs "by accident" rather than having anything to do with being distributed. And I'd choose Git in a heartbeat for virtually any project over something like Subversion because of those two reasons, but I definitely disagree with the assessment that distributed is the only way forward.

[Rysto]

I have to use Clearcase at work(although I'm trying to change that). To me, a significant advantage of DVCS is never being brought to a standstill because the central server is down.

Plus, if too many operations go through the server it becomes a scalabilty nightmare.

Edit: Oh, and if your business expands to the point where you have a second site half a world away, your remote workers will *not* appreciate having to do all of their work through a sever on the other side of a VPN.

[Steax]

I definitely disagree with the assessment that distributed is the only way forward.

I'm thinking it's the way forward due to the nature of how collaboration is moving. We're constantly edging towards a work environment where people can work from anywhere, where on-a-whim brilliant ideas at 2 AM are encouraged, and data is increasingly moving to the "cloud" rather than a big server in an office. While I agree that a DVCS, at this point, still loses out at certain points against non-distributed ones, I do think it's the only way forward.

[Sc4Freak]

Also, DVCSs tend to be awfully slow at handling huge amounts of large binary files. Which is extremely common in some domains like game development.

[Thesh]

If there was a centralized repository that allowed you to create checkpoints that other devs don't get (while still being saved on a centralized, backed up server) and was based on changesets, I think it would be more versatile.

Until you want to work on the bus or plane.

Though I do agree that, for a centralized-organization-type project where the version control doesn't present any new access issues, most of the benefits of Git over Subversion are "artifical" in the sense that a centralized VCS could provide the appropriate features (e.g. something like git add --patch/--interactive), but they just don't for whatever reason.

At that point, allowing you to create checkpoints offline would suffice. It's not every day that you need to go back and view revision history or create a new branch. Even then, creating local branches offline wouldn't be a problem, you don't need the whole revision history for that.

1. Local commits are faster.

You still need to push to a centralized repo often so that people can get your work. The only difference between SVN and Git in this regard are how often you commit (with SVN I find you either commit too often, or not often enough - with Git you either commit as often as you should... or not often enough). Adding a checkpointing system like I mentioned would solve this.

2. Local clone means you can work from anywhere.

Provided you have a laptop with your repo on it to bring with you. If you don't, with a centralized version control you can still get everything that you have committed. If you add centralized checkpointing, you would be able to work from any machine provided you can access the central server, and then get all the work that you have created checkpoints for (whether or not you have committed it).

[Aaeriele]

1. Local commits are faster.

You still need to push to a centralized repo often so that people can get your work. The only difference between SVN and Git in this regard are how often you commit (with SVN I find you either commit too often, or not often enough - with Git you either commit as often as you should... or not often enough). Adding a checkpointing system like I mentioned would solve this.

At which point you've effectively created a DVCS anyway.

2. Local clone means you can work from anywhere.

Provided you have a laptop with your repo on it to bring with you.

Er, yes? That seems like a reasonable assumption for just about any remote work case.

[EvanED]

1. Local commits are faster.

You still need to push to a centralized repo often so that people can get your work. The only difference between SVN and Git in this regard are how often you commit (with SVN I find you either commit too often, or not often enough - with Git you either commit as often as you should... or not often enough). Adding a checkpointing system like I mentioned would solve this.

At which point you've effectively created a DVCS anyway.

I definitely wouldn't call it a DVCS if you don't have a local copy of the repository you can work from. Your disconnected operation is very impoverished in terms of the possible feature set, but still very useful because of how often (or really, how not often) most of the features are used.

This is pretty much what I was thinking when I mentioned a "hybrid" system earlier.

[Xeio]

Code: Select all

//Uh, oh.

[Aaeriele]

I definitely wouldn't call it a DVCS if you don't have a local copy of the repository you can work from. Your disconnected operation is very impoverished in terms of the possible feature set, but still very useful because of how often (or really, how not often) most of the features are used.

I was more just pointing out that saving local snapshots is just a hop skip and a jump away from implementing full disconnected operation, because it requires you to be able to do local history tracking - once you have that, adding the option to download more than just the latest version is pretty trivial.

[EvanED]

I have to use Clearcase at work(although I'm trying to change that). To me, a significant advantage of DVCS is never being brought to a standstill because the central server is down.

While I'm somewhat sympathetic to this, it's also something that doesn't really arise in my environment: if the "Subversion server" (not that we really have such a thing) goes down, it's because the network file system (we use AFS) is down. And if AFS is down, you wouldn't be able to work "locally" anyway.

Edit: Oh, and if your business expands to the point where you have a second site half a world away, your remote workers will *not* appreciate having to do all of their work through a sever on the other side of a VPN.

and

I definitely disagree with the assessment that distributed is the only way forward.

I'm thinking it's the way forward due to the nature of how collaboration is moving. We're constantly edging towards a work environment where people can work from anywhere, where on-a-whim brilliant ideas at 2 AM are encouraged, and data is increasingly moving to the "cloud" rather than a big server in an office. While I agree that a DVCS, at this point, still loses out at certain points against non-distributed ones, I do think it's the only way forward.

Like I said, I'm not saying that centralized VCSs are where it's at, just that... they get a really bad rap in my mind. Again I'd say that you could set up a hybrid system that only talks to the remote server when necessary (commits) and otherwise has a mirror.

But look at software companies today. Employees of even "progressive" software companies still do most work in the office.

I definitely wouldn't call it a DVCS if you don't have a local copy of the repository you can work from. Your disconnected operation is very impoverished in terms of the possible feature set, but still very useful because of how often (or really, how not often) most of the features are used.

I was more just pointing out that saving local snapshots is just a hop skip and a jump away from implementing full disconnected operation, because it requires you to be able to do local history tracking - once you have that, adding the option to download more than just the latest version is pretty trivial.

Hmm, I'm not sure I agree with this. For instance, if you don't want to do something like git rerere, you could imagine a set of somewhat lightweight wrapper scripts around Subversion that provide that level of disconnected operation. (svn-wrap commit would just store a new local copy of the present tree status, and svn-wrap diff would diff vs that. If you wanted to push commits on a 1:1 basis instead of in one chunk, svn-wrap push would just replay those diffs.) Writing a wrapper to allow full disconnected operation would be far harder. (Or at least far more involved.)

Don't forget, DVCSs nearly-fundamentally impose significant restrictions on repository layout. In particular, they preclude having one big-ass repository. And while such a repository has a lot of disadvantages, it has a lot of advantages too: most notably, if you're working on a project that involves several, you only have one thing to worry about rather than "I need version 10 of repository A, version 35 of repository B, version 52 of repository C, etc."

[Steax]

FT: I really need a proper MySQL versioning system. Also preferably a MySQL backup script (does that necessitate a full dump?). Any suggestions?

[headprogrammingczar]

FT: I really need a proper MySQL versioning system. Also preferably a MySQL backup script (does that necessitate a full dump?). Any suggestions?

Code: Select all

mysqldump -u user --password=password database > /path/to/dump.sql
cp /path/to/dump.sql /path/to/backup/dir

The dump size will usually be smaller than the memory footprint of the database itself. If regular dumps are performance-prohibitive, you can also try mysqlhotcopy, binary logs, or even go all the way and hot-swap RAID drives (which will back up everything else, but hey...).

See also, the mysql docs.

I'm not sure what you mean by versioning system, so I can't comment on that part of the question.

[Aaeriele]

I'd assume something like a database migration library. That depends on what you're working with (e.g. Django has South).

[bittyx]

@Steax: I've never actually used it, but I have this in my bookmarks for if I ever need it: http://schemasync.org/

You could also use something like this: http://codeigniter.com/user_guide/libra ... ation.html <--- I've been meaning to start implementing something like this for a while now, but haven't gotten around to it. Seems pretty convenient in theory, but as I haven't tried it, I can't speak from experience.

[You, sir, name?]

So, I left an ugly spaghetti code ad-hoc solution in the code a while back. "I'll get around to fixing it", I told myself. Today I needed to change it, and the noodles had like turned into tentacles, and the code had turned into some sort of demented eldritch abomination.

Spoiler:

Tru fax.

[Jplus]

Spaghetti code, you say? Then where are the unguarded jumps?

[Steax]

FT: I really need a proper MySQL versioning system. Also preferably a MySQL backup script (does that necessitate a full dump?). Any suggestions?

Code: Select all

mysqldump -u user --password=password database > /path/to/dump.sql
cp /path/to/dump.sql /path/to/backup/dir

The dump size will usually be smaller than the memory footprint of the database itself. If regular dumps are performance-prohibitive, you can also try mysqlhotcopy, binary logs, or even go all the way and hot-swap RAID drives (which will back up everything else, but hey...).

See also, the mysql docs.

I'm not sure what you mean by versioning system, so I can't comment on that part of the question.

The docs explain incremental backups, I think that's what I want. Running backups too often tends to cause the files to add up.

I'd assume something like a database migration library. That depends on what you're working with (e.g. Django has South).

Exactly like that, but I need one for PHP. Digging time.

@Steax: I've never actually used it, but I have this in my bookmarks for if I ever need it: http://schemasync.org/

You could also use something like this: http://codeigniter.com/user_guide/libra ... ation.html <--- I've been meaning to start implementing something like this for a while now, but haven't gotten around to it. Seems pretty convenient in theory, but as I haven't tried it, I can't speak from experience.

The CodeIgniter one is more or less exactly what I need, except I don't use CodeIgniter. The goal is the same; the need to automatically update database schemas without having to manually throw around SQL scripts.

Thanks everyone!

[You, sir, name?]

Spaghetti code, you say? Then where are the unguarded jumps?

Not quite at go-to, but it had a few too many named breaks for comfort.

[Shivahn]

Coding fleeting thought: What does it mean to know a language? Like, when do you feel comfortable saying "I know Ruby" or something? Or is it just something you go with when you feel confident in your ability to code an arbitrary thing?

[Thesh]

Coding fleeting thought: What does it mean to know a language? Like, when do you feel comfortable saying "I know Ruby" or something? Or is it just something you go with when you feel confident in your ability to code an arbitrary thing?

I think you can say that you know a language when you don't have to look up syntax, and spend more time writing code than looking up built-in functions when writing a program.

[Shivahn]

That seems like a pretty good definition. Cool!

[Steax]

I generally assume that someone who says that also understands the common patterns and use cases of said language. By "knowing" a language, they should be able to more or less interpret raw code you give them.

[Jplus]

I'm afraid it depends on the language. For example in C++, knowing the syntax and the standard library functions certainly isn't enough to say that you know the language. I passed that point, but there turns out to be more to learn every day. OOP, generic programming and template metaprogramming are all extremely rich topics with aspects that are very specific to C++. Not to mention several Boost libraries that require entirely new ways of thinking.

On the other hand, I think I spent less than 50 hours with Lua but I may be bold enough to say that I know it already. Even though I regularly need to have a look at the cheat sheet.

[troyp]

I'm afraid it depends on the language. For example in C++, knowing the syntax and the standard library functions certainly isn't enough to say that you know the language. I passed that point, but there turns out to be more to learn every day. OOP, generic programming and template metaprogramming are all extremely rich topics with aspects that are very specific to C++. Not to mention several Boost libraries that require entirely new ways of thinking.

But surely if someone asked if you knew C++, you'd reply "yes"?

C++ is probably an exception in many ways, being such a monstrous huge language. My impression (not being a C++ coder) is that most people tend to use a specific subset of it and may become quite competent even while having almost no knowledge of a significant fraction of the language.

edit: s/the it/it/

[Steax]

I think that's typically why people will say they know a brand/framework/library/whatnot instead of just "I know C++". Anyone plain-out saying that is likely either grossly underestimating the language, or a really really good programmer.

[Thesh]

I think that's typically why people will say they know a brand/framework/library/whatnot instead of just "I know C++". Anyone plain-out saying that is likely either grossly underestimating the language, or a really really good programmer.

I disagree. Knowing a language doesn't mean you're a good or experienced programmer in that language, it just means you can program in it. What you are getting at, in my mind, is the difference between saying "I know C++" and "I am a C++ programmer".

[Steax]

I was implying that in the case of C++. As in, someone saying that they "know" it is either a really confident person who we'd all probably know by name anyway (making the statement unnecessary) or an overconfident person who doesn't understand the scale of C++ (making the statement meaningless).

So I do basically agree that just because you know a language, you don't qualify as a good programmer in it.

My language skills need work.

[Thesh]

So apparently, .NET implements PBKDF1 which allows you to specify a hash algorithm, and PBKDF2 using HMAC-SHA1. I need PBKDF2 using HMAC-SHA384, which means I have to write my own implementation (or steal someone else's). For the level of abstraction that System.Security.Cryptography has, is a generic PBKDF2 class too much to ask for?

[Aaeriele]

For the level of abstraction that System.Security.Cryptography has, is a generic PBKDF2 class too much to ask for?

Apparently, yes.

[Thesh]

I believe Thesh may have been asking a rhetorical question.

[headprogrammingczar]

Coding fleeting thought: What does it mean to know a language? Like, when do you feel comfortable saying "I know Ruby" or something? Or is it just something you go with when you feel confident in your ability to code an arbitrary thing?

I would say you know a language when you have nothing left to learn about it. For instance, the last few steps to knowing Java would be generics, static classes, design patterns. In C, you would be done when you have learned POSIX, function pointers, and maybe a bit of assembly.

[troyp]

Coding fleeting thought: What does it mean to know a language? Like, when do you feel comfortable saying "I know Ruby" or something? Or is it just something you go with when you feel confident in your ability to code an arbitrary thing?

I think we've established that you should never feel comfortable saying "I know Ruby" as the term is too vague and none of us can agree on what it means.

[Thesh]

So apparently, .NET implements PBKDF1 which allows you to specify a hash algorithm, and PBKDF2 using HMAC-SHA1. I need PBKDF2 using HMAC-SHA384, which means I have to write my own implementation (or steal someone else's). For the level of abstraction that System.Security.Cryptography has, is a generic PBKDF2 class too much to ask for?

And 70 lines of code later, I have a working generic, static Pbkdf2 class. See, microsoft, it's that easy.

[Shivahn]

Yeah, "know" is a bit of an ambiguous word. But people say "Oh, I know C++, Lisp, and Python" all the time, and I was wondering what connotation that usually should give.

But I realize that depending on the person, that could mean they know every in and out, or "I can code FizzBuzz in these languages."