## Forum Migration

The school experience. School related queries, discussions, and stories that aren't specific to a subject.

Moderators: gmalivuk, Moderators General, Prelates

Posts: 1362
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

### Re: Forum Migration

Safari on iphone6
It worked before migration to stay logged in and other sites are still functioning in this manner so i dont think its a phone setting.
I've noticed there is now a section dedicated to autologin keys in the UCP. Maybe try deleting all keys there?

The Snide Sniper
Posts: 65
Joined: Mon Mar 21, 2016 7:25 pm UTC
Location: UTC-6:00

### Re: Forum Migration

So, if the forum got upgraded to a later software version, then what version is it?
Also known as "RamenChef" on various other fora.
Me on: Drawception | Explain xkcd | NationStates | The Sluggite Zone | Stack Overflow

Flumble
Yes Man
Posts: 2076
Joined: Sun Aug 05, 2012 9:35 pm UTC

### Re: Forum Migration

Yay, moderner forum software. I, for one, welcome the new mobile layout too.

I do wonder if davean is familiar with the seperation of testing and production environment. Having the forum inaccessible for more than an hour, to me, is very unnerving, especially during Madness. (or are the 503s I had the whole day unrelated to the upgrade?)

gmalivuk
GNU Terry Pratchett
Posts: 26529
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Forum Migration

Did you read the OP? I suspect the inaccessibility might have had something to do with a physical hardware change.
davean wrote:I just migrated the forums to new hardware. DNS will take a few hours to settle. Please let me know if you see any new issues.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Giant Speck
Bouncy Sex Marshmallow
Posts: 3808
Joined: Tue Sep 08, 2009 12:30 pm UTC
Location: Tucson, Arizona

### Re: Forum Migration

The Snide Sniper wrote:So, if the forum got upgraded to a later software version, then what version is it?

My guess would be 3.1.8, which was released in February.
"Did I say recently that I love Giant Speck? Because I love Giant Speck. He is the best." - Weeks

davean
Site Ninja
Posts: 2498
Joined: Sat Apr 08, 2006 7:50 am UTC
Contact:

### Re: Forum Migration

Flumble wrote:I do wonder if davean is familiar with the seperation of testing and production environment. Having the forum inaccessible for more than an hour, to me, is very unnerving, especially during Madness. (or are the 503s I had the whole day unrelated to the upgrade?)

What? You wanted a set of posts that people posted that suddenly disappeared? What does this have to do with testing?

SecondTalon
SexyTalon
Posts: 26279
Joined: Sat May 05, 2007 2:10 pm UTC
Location: Louisville, Kentucky, USA, Mars. HA!
Contact:

### Re: Forum Migration

More with Flumble not understanding the change, I reckon. Whether ignorance or inattentiveness I can't say.
heuristically_alone wrote:I want to write a DnD campaign and play it by myself and DM it myself.
heuristically_alone wrote:I have been informed that this is called writing a book.

Thesh
Posts: 6301
Joined: Tue Jan 12, 2010 1:55 am UTC

### Re: Forum Migration

I think we should be get a refund for our membership fees, prorated for any outage beyond the SLA.
Summum ius, summa iniuria.

Xanthir
My HERO!!!
Posts: 5334
Joined: Tue Feb 20, 2007 12:49 am UTC
Contact:

### Re: Forum Migration

Thesh wrote:Most browsers these days support MathML (HTML5 [itex] tags)

Untrue. Only Firefox and Safari support it. All us Chrome and IE users would just see jumbled text.

(That said, MathJax is *really good* in my experience.)
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

Flumble
Yes Man
Posts: 2076
Joined: Sun Aug 05, 2012 9:35 pm UTC

### Re: Forum Migration

SecondTalon wrote:More with Flumble not understanding the change, I reckon. Whether ignorance or inattentiveness I can't say.

It seems I just don't get it. My observations yesterday were: first 503s all day (and even the evening before IIRC), then the "you've successfully installed apache" page for some time and finally the 'new' forum.
Based on the former two, my assumption is either that davean was spreading the migration over a whole day with downtime that my inner sysadmin abhors or the 503s are an unrelated issue.
In any case, I'm more confused now than I was before your replies.

Thesh wrote:I think we should be get a refund for our membership fees, prorated for any outage beyond the SLA.

My Madness was cut short this year; how do I apply?

Soupspoon
You have done something you shouldn't. Or are about to.
Posts: 3682
Joined: Thu Jan 28, 2016 7:00 pm UTC
Location: 53-1

### Re: Forum Migration

Flumble wrote:In any case, I'm more confused now than I was before your replies.

The way I saw it (BICBW), the migration didn't precipitate the errors, the errors precipitated the migration. Or, at least, moved the timescale forward on a previously planned project.

Hardware failure a couple of days ago forced a reboot and maybe a well-placed physical blow to stop the fan from making a rattling noise. But new hardware was then sought (if not already present and reserved for this, either to be newly bought in or else discovered in amongst whatever Aladdin's Cave of computer-hardware delights davean had routine and free access to salvage from).

It then failed again yesterday, and as the emergency process of setting up the new server was already started (and/or the old hardware didn't this time recover by the old switch-it-off-then-on-again of the previous circadian cycle) the 'new look' forum was finished off, prior data imported and made live as quickly as possible, under the circumstances. (And you just happened to be camping on it at the point that the basic server software first came online, but before the php/forum backend elements were yet installed.)

But that's just a guess. And it doesn't really matter so long as we appreciate that effort was put in towards rectifying the problems, regardless of what form the effort actually consisted of.
Last edited by Soupspoon on Wed Mar 30, 2016 4:14 pm UTC, edited 1 time in total.

gmalivuk
GNU Terry Pratchett
Posts: 26529
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Forum Migration

Flumble wrote:
SecondTalon wrote:More with Flumble not understanding the change, I reckon. Whether ignorance or inattentiveness I can't say.

It seems I just don't get it. My observations yesterday were: first 503s all day (and even the evening before IIRC), then the "you've successfully installed apache" page for some time and finally the 'new' forum.
Based on the former two, my assumption is either that davean was spreading the migration over a whole day with downtime that my inner sysadmin abhors or the 503s are an unrelated issue.
In any case, I'm more confused now than I was before your replies.

How would you have done it, and how long do you think that would have taken?
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Thesh
Posts: 6301
Joined: Tue Jan 12, 2010 1:55 am UTC

### Re: Forum Migration

Given that no details have been given, it's probably hard for someone to say what they would have done differently. As far as I can tell, there was both a new server and new software, and if that software update has database structure changes, well, then there isn't a lot you can do without significant development work. But really, boo fucking who, the ad-free, subscription-free website was down for half a day. You made it through, suck it up.
Summum ius, summa iniuria.

davean
Site Ninja
Posts: 2498
Joined: Sat Apr 08, 2006 7:50 am UTC
Contact:

### Re: Forum Migration

Lets see, dumping the DB from postgresql, across SSH, into pgsql took just under 4 hours. Then rsyncing the forum uploads across took another 2 hours, but I did that before taking the forums down, updating to to catch the changes from between rsyncing and shutting down the old webserver was about 15 minutes. Running the migrations for the upgrade was about 16 hours. Cleaning the DB up after the migration (things like old mods, etc) was 30-ish minutes.

Thats not everything in the migration, but its a lot of what had to happen sequentially.

Wildcard
Candlestick!
Posts: 253
Joined: Wed Jul 02, 2008 12:42 am UTC
Location: Outside of the box

### Re: Forum Migration

Yay for migration. (Boo for hamburgers.)

The screen width is irritating, but the only killer for me is the erratic-ness of the egosearch.

When I first opened it up, it showed only one post. (???)

When I changed the search scope from "All results" to "1 year" I got a nice big page full of results. When I changed it back to "All results", I got one result again. Back to "1 year" and I got only *five* results. Change to "6 months" and I get a page full. Very erratic; it continues to change with mesmerizing kaleidoscopicness.

(By the way, if the new version **supports standard markdown**, I *don't care* about things like egosearch. Markdown is the _key_ to my heart. ...eh, pity.)
There's no such thing as a funny sig.

Thesh
Posts: 6301
Joined: Tue Jan 12, 2010 1:55 am UTC

### Re: Forum Migration

I helped plan a cross-datacenter migration for a SaaS company. What we did was take a backup of the database, copy the backup across datacenters, restore the backup, put the software in read-only mode, and then took a differential backup (which was much faster, and much smaller), copied it over and restored the differential. The software at the old datacenter stayed up in read-only mode until the DNS was propagated. I have no clue what kind of facilities PostgreSQL has for backups.

However, being that most of it was the upgrade process, you are kind of SOL.
Summum ius, summa iniuria.

flicky1991
Like in Cinderella?
Posts: 756
Joined: Fri Feb 11, 2011 3:36 pm UTC
Location: London

### Re: Forum Migration

I like some of the changes, but I'll only be truly happy when egosearch works again.
any pronouns
----
Forum Games Discord
(tell me if link doesn't work)
----
NiteSciFi, sci-fi forum that needs more members

Flumble
Yes Man
Posts: 2076
Joined: Sun Aug 05, 2012 9:35 pm UTC

### Re: Forum Migration

davean wrote:Lets see, dumping the DB from postgresql, across SSH, into pgsql took just under 4 hours. Then rsyncing the forum uploads across took another 2 hours, but I did that before taking the forums down, updating to to catch the changes from between rsyncing and shutting down the old webserver was about 15 minutes. Running the migrations for the upgrade was about 16 hours. Cleaning the DB up after the migration (things like old mods, etc) was 30-ish minutes.

Thats not everything in the migration, but its a lot of what had to happen sequentially.

Thanks for the detailed overview!
Wow, I didn't expect the conversion to take 16 hours. And I understand that leaving the old forum in read-only mode (assuming it even exists) for that long is pretty much useless.

gmalivuk wrote:How would you have done it, and how long do you think that would have taken?

As it turns out, what davean has done and it would take about 16 hours of unavailability. (though I would keep the old forum in read-only mode until the migration is complete, preferably with DON'T PANIC a message saying the forum is getting upgraded in big friendly letters)

Basically what I was missing during the migration was a reassuring message that the forum hasn't died of Madness and taking the whole database with it, but rather that it's all part of the plan.

gmalivuk
GNU Terry Pratchett
Posts: 26529
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Forum Migration

so is egosearch well and truly gone?
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Azrael
CATS. CATS ARE NICE.
Posts: 6491
Joined: Thu Apr 26, 2007 1:16 am UTC
Location: Boston

### Re: Forum Migration

doogly
Dr. The Juggernaut of Touching Himself
Posts: 5454
Joined: Mon Oct 23, 2006 2:31 am UTC
Location: Lexington, MA
Contact:

### Re: Forum Migration

So it's there, but it doesn't work at all. It thinks I have posted in only two threads.
LE4dGOLEM: What's a Doug?
Noc: A larval Doogly. They grow the tail and stinger upon reaching adulthood.

Keep waggling your butt brows Brothers.
Or; Is that your eye butthairs?

gmalivuk
GNU Terry Pratchett
Posts: 26529
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Forum Migration

I know the url still exists. Does yours have more than one thread in it, Az?
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Soupspoon
You have done something you shouldn't. Or are about to.
Posts: 3682
Joined: Thu Jan 28, 2016 7:00 pm UTC
Location: 53-1

### Re: Forum Migration

doogly wrote:So it's there, but it doesn't work at all. It thinks I have posted in only two threads.

Discussed a few messages up this thread, and I've just been experimenting myself. If you change the time limit, at the bottom (and [Go] with it) it seems to work better. I've not been on that long, so once I've got it to one year it'll more than cover everything I do and (if still not unstable) might be better than the setting of any time at all...

doogly
Dr. The Juggernaut of Touching Himself
Posts: 5454
Joined: Mon Oct 23, 2006 2:31 am UTC
Location: Lexington, MA
Contact:

### Re: Forum Migration

Oh hey! That's a thing.
LE4dGOLEM: What's a Doug?
Noc: A larval Doogly. They grow the tail and stinger upon reaching adulthood.

Keep waggling your butt brows Brothers.
Or; Is that your eye butthairs?

flicky1991
Like in Cinderella?
Posts: 756
Joined: Fri Feb 11, 2011 3:36 pm UTC
Location: London

### Re: Forum Migration

Oh wow, yes, setting to 1 year fixes it. Perfect.
any pronouns
----
Forum Games Discord
(tell me if link doesn't work)
----
NiteSciFi, sci-fi forum that needs more members

gmalivuk
GNU Terry Pratchett
Posts: 26529
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Forum Migration

When I change it to something else, it starts out showing a whole page of results, but the number of results reported is much smaller, and when I go back to that same results page (like after viewing a thread) it only shows that smaller number of results.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Posts: 2755
Joined: Fri Feb 20, 2009 12:35 pm UTC
Location: 한국 창원

### Re: Forum Migration

It seems like you can get better results by changing the search options (not just the time limit, changing the sorting options such as Author, Post Time, &c. seems to have a similar effect), but it's very erratic and there's clearly still something strange going on. With '1 year' selected, I first got a full page of results, but now when I take the same steps I only see 5 results. Searching by 'Author' gave me a full page originally, but now only a single result.

It's almost like every option gives you a 'one time' view of the whole list, but then if you use that option again, it breaks.

I also note that if I change the search to 'Ascending' rather than 'Descending' it always gives me a full page, but with the message "Search found 1 match • Page 0 of 1"

flicky1991
Like in Cinderella?
Posts: 756
Joined: Fri Feb 11, 2011 3:36 pm UTC
Location: London

### Re: Forum Migration

gmalivuk wrote:When I change it to something else, it starts out showing a whole page of results, but the number of results reported is much smaller, and when I go back to that same results page (like after viewing a thread) it only shows that smaller number of results.

Yep, same.
any pronouns
----
Forum Games Discord
(tell me if link doesn't work)
----
NiteSciFi, sci-fi forum that needs more members

Soupspoon
You have done something you shouldn't. Or are about to.
Posts: 3682
Joined: Thu Jan 28, 2016 7:00 pm UTC
Location: 53-1

### Re: Forum Migration

[@Flicky, but ninjaed... But as I put the results together, posting anyway in case it's any use.]

Yes, but I can confirm it's unreliable:

All results: 9 (was 8, when reported yesterday; doesn't now include one newer added-to-thread, but does include another)
1 day: 1 (Forum Migration Thread should be at least 4!)
7 days: 4 (less than last time I used 7 days; all displayed ones also likely to be "1 day" candidates)
2 weeks: 1 (!)
1 month: 9 (?)
3 months: 9 (??)
6 months: 50 (I didn't register more than 3 months ago, so this upper limit should apply to the last option too...)
1 year: 50 (...and now I'm going back up the list again)
6 months: 9 (different from first pass!)
3 months: 9
1 month: 9
2 weeks: 1 (Forum Migration thread, only)
7 days: 4
1 day: 1 (again).
All results: 9

So... a certain symmetry, if confusingly so. But "6 months" displayed different between passes, and recorded passes look different from my first experimentation. (I'm willing to believe that 50 threads is my true upper limit, though, until I later on discover more.)

If it was just the 1 day (and, for some strange reason "All results") that was wrong I'd hypothesise that the only thread in there (Form Migration Thread) is the only thread created since migration, so for some reason only gets counted because of that, and the All Results one (perhaps because it isn't explicit in the time-choice, defaults to some sort of "since migration only" matching, in the background). And/or combined with "only threads created since \$time_limit", regardless of whether personal contributions were made within said limit. But none of that explains the count-reduction reduction between 7 days and 2 weeks...

(ETA: Note that I that the "Tire Swing" comic thread is a thread that (at least at some times) not appeared in all relevant Egosearches, despite there being a newer message marked in it when I go to the Individual Comics and (now) adjust the Egosearch to time-limits that display it. It's a thread from just before the migration, I think, which is still active but is volatile insofar as egosearch listing, as described.)

(ETA2: And "All results" just now reverted back to 8 (the 'classic' 9 - that never included the Tire Swing thread, but now minus the recent XKCD Skyskraper thread from the games subforum, SFAICT), so an improvement on just Forum Migration, but... odd. But a datum point.)
Last edited by Soupspoon on Fri Mar 25, 2016 1:02 pm UTC, edited 2 times in total.

Dauric
Posts: 3943
Joined: Wed Aug 05, 2009 6:58 pm UTC
Location: In midair, traversing laterally over a container of sharks. No water, just sharks, with lasers.

### Re: Forum Migration

Adacore wrote:It seems like you can get better results by changing the search options (not just the time limit, changing the sorting options such as Author, Post Time, &c. seems to have a similar effect), but it's very erratic and there's clearly still something strange going on. With '1 year' selected, I first got a full page of results, but now when I take the same steps I only see 5 results. Searching by 'Author' gave me a full page originally, but now only a single result.

It's almost like every option gives you a 'one time' view of the whole list, but then if you use that option again, it breaks.

I also note that if I change the search to 'Ascending' rather than 'Descending' it always gives me a full page, but with the message "Search found 1 match • Page 0 of 1"

Seeing pretty much the same myself. First time I search by 1-year I see one page beginning with last year's Mod Madness announcement thread, when I search by 1-year again i get only three entries beginning with this year's Mod Madness alert. I I search by "All Results" I get only the "On other News" thread (News and Articles forum)
We're in the traffic-chopper over the XKCD boards where there's been a thread-derailment. A Liquified Godwin spill has evacuated threads in a fourty-post radius of the accident, Lolcats and TVTropes have broken free of their containers. It is believed that the Point has perished.

doogly
Dr. The Juggernaut of Touching Himself
Posts: 5454
Joined: Mon Oct 23, 2006 2:31 am UTC
Location: Lexington, MA
Contact:

### Re: Forum Migration

Soupspoon wrote:[@Flicky, but ninjaed... But as I put the results together, posting anyway in case it's any use.]

Yes, but I can confirm it's unreliable:

Yeah, it caught on to my workaround and has put in new countermeasures to thwart my desires.
LE4dGOLEM: What's a Doug?
Noc: A larval Doogly. They grow the tail and stinger upon reaching adulthood.

Keep waggling your butt brows Brothers.
Or; Is that your eye butthairs?

patzer
Posts: 429
Joined: Fri Mar 30, 2012 5:48 pm UTC
Contact:

### Re: Forum Migration

Umm, egosearch is really easy now. I think.

e.g.

davean
Site Ninja
Posts: 2477

That "2477" is now a hyperlink. Click on it, and you get all of davean's posts.
If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family Anatidae on our hands. –Douglas Adams

flicky1991
Like in Cinderella?
Posts: 756
Joined: Fri Feb 11, 2011 3:36 pm UTC
Location: London

### Re: Forum Migration

That doesn't tell you which of the threads you've posted in have new replies, which is the main function of egosearch.
any pronouns
----
Forum Games Discord
(tell me if link doesn't work)
----
NiteSciFi, sci-fi forum that needs more members

f5r5e5d
Posts: 104
Joined: Tue May 08, 2012 3:22 am UTC

### Re: Forum Migration

how do I get to the current comic page from the forum? - "xkcd" in the banner doesn't appear to be a link (IE11)

Dauric
Posts: 3943
Joined: Wed Aug 05, 2009 6:58 pm UTC
Location: In midair, traversing laterally over a container of sharks. No water, just sharks, with lasers.

### Re: Forum Migration

patzer wrote:Umm, egosearch is really easy now. I think.

e.g.

davean
Site Ninja
Posts: 2477

That "2477" is now a hyperlink. Click on it, and you get all of davean's posts.

It gives you all the individual posts, but doing an Egosearch just gives you the threads. To wit: If I look at my post count I get 88 pages of posts, where I only have 20 pages of threads I've posted in. It's easier to search for a specific topic with the shorter list of threads than it is to dig through all the individual posts one has ever made.
We're in the traffic-chopper over the XKCD boards where there's been a thread-derailment. A Liquified Godwin spill has evacuated threads in a fourty-post radius of the accident, Lolcats and TVTropes have broken free of their containers. It is believed that the Point has perished.

gmalivuk
GNU Terry Pratchett
Posts: 26529
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Forum Migration

patzer wrote:Umm, egosearch is really easy now. I think.

As others have explained, we want the threads, not the posts. We're calling it "egosearch" because it's what "search_id=egosearch" is supposed to give.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

rmsgrey
Posts: 3465
Joined: Wed Nov 16, 2011 6:35 pm UTC

### Re: Forum Migration

gmalivuk wrote:
patzer wrote:Umm, egosearch is really easy now. I think.

As others have explained, we want the threads, not the posts. We're calling it "egosearch" because it's what "search_id=egosearch" is supposed to give.

My egosearch is only 3 posts.

I'm also having trouble with the "jump to first unread post" functionality - instead of landing on the first unread post, it lands a (variable) little way above - possibly it's going to the right place but then loading more images that changes the correct height, and not updating to reflect that...

K-R
Posts: 1564
Joined: Thu Mar 15, 2012 6:42 pm UTC
Location: Australia

### Re: Forum Migration

Yeah, I've had the same a couple of times, always landing me at the start of the last read post instead of the first unread post.

davean
Site Ninja
Posts: 2498
Joined: Sat Apr 08, 2006 7:50 am UTC
Contact:

### Re: Forum Migration

Sub/Sup should now be working again.

As for egosearch I'm looking into it but am mostly confused. Search in general seems to be a bit weird.

BlitzGirl
Posts: 8999
Joined: Mon Sep 20, 2010 11:48 am UTC
Location: Both Present and Past...... Schizoblitz: 115/2601 NP
Contact:

### Re: Forum Migration

Thanks for fixing sub/sup! Our footnotes1 thank you too!

1and there are a lot of footnotes