Here comes the storm!

[tffy]

The e-mail I got earlier today was a bit of a surprise. It basically said: We've devised a brand-new system to fairly give you your '16 ticket! A lottery! Aren't we smaaaaart! It went so good in '12!

Solution (perhaps read this after the sh*t storm that will happen in about 14 hours dies down?) should be comprised of two parts.

1. The bumrush that happens isn't that major by the real-world standards, I don't think. I'm not a network engineer, but... Grateful Dead reunion and Tool 'reunion' concerts come to mind. Why haven't you, in the 4 years that this BS has been happening, haven't figured out how to deal with it? (changing the people in charge of dealing with it would be my first suggestion). Changing the ticket sales provider that you guys are for some reason inextricably linked with...well, one name comes to mind... is a second one?

Dealing with server overload/congestion/traffic is a pretty well-explored area in this modern internet world. Your overload is not that heavy, I don't think. There are ways to rent server capacity for cases just like this. Why haven't you fixed it?

2. Changing the number of tickets each registrant can buy from 2 to 1 could also be done. I know, sucks for us all - but that, WOULD increase the chances - for an individual who put in the effort - of getting a ticket basically by a factor of close to 2.

Thoughts?

-Sherpa

[BBadger]

This isn't like building computer hardware to do microsecond-speed transactions for the stock market where you might actually make some money. If it mattered so much, people would buy better hardware and site that hardware close to the servers selling the tickets, so that they could snipe tickets like people do with eBay.

No, it's not worth the time and money to buy additional servers and delivery networks, and synchronize them all to retain the first-come-first-served ordering, just to service a bunch of people trying to buy a few tens of thousands of tickets for some festival in the desert.

[tffy]

Hence the word [b]"rent"[/b] that I used. Big boys like Amazon, who have spend a lot of money and smart-people-time on developing all the fast fancy high-power servers... they sell them, you know... and you don't gotta buy them for a whole year.

Also, since the goal is simply to give the system ample power to not crash when this piddly load is placed on it... and the system isn't going to be fighting some 'evil money-making baddie who'll make .00001 cent 8 billion times'... you don't even have to synchronize nothin' to nothin - nobody will know that the server that took the order from 12:00:00:015 and the one that took the order from 12:00:00:350 actually traded places because of the length of their wires. The guys who pushed 'buy' certainly won't know. Noone will care. People will be happy enough that their shit didn't crash for an hour in a stupid up and down and up and down and oh sorry we're sold out line. Uuuggghhhhh...

It is exactly that - the fact that the tech problem is so easy to solve and that is has not gotten solved for so long is the part that baffles me.

[Jackass]

Servers and time-stamps will mean nothing tomorrow... Here, pick one

[tffy]

Uuuhhhh, yeaaaahh. Just like in '12.

Read up on people's overwhelming happiness with how THAT turned out.

Maybe BM is banking on well, we've distributed some "core group" tickets this time around so the guys/gals who are important won't raise the stink they did in '12. That's very likely what it is, actually. Maybe the stink will be a lot less.

Just funny that after 3 years of saying "Well that went baaaaad in '12" they (and on the last day before the raffle, mind you...) decided to give it a good-old-try one more time.

[Jovankat]

No matter how much money they spend on ticket selling tech there are still going to be people who don't get tickets who want them. More servers wouldn't solve the supply and demand issue. As less than completely optimal as the ticket selling process may be it does manage to get 30,000 tickets sold to people who want them in an hour. It works. Why throw money at it to change who the 30,000 people who get the tickets are when they could instead spend the money on initiatives that might allow them to increase the population cap and sell more tickets next year. Or should they spend less on the art that makes that week in the desert so awesome to try and make that hour of ticket sale slightly less annoying?

It's about bang for buck and clearly they don't think the improvements that could be made are worth the money and/or they feel there are more important things to spend money on.

[tffy]

In my book, works needs to stand for a system that is: transparent, not susceptible to subterfuge by hackers and one that does not hang and crash. The jury is still out on the year '16, but in the years 12-15, the system DID NOT WORK ONE BIT.

So let's base our arguments on past facts and not on rosy future projections, alright?

With that in mind, projections about the 'incredible tons of untold internet dollars' needed to solve a once-a-year 100,000 separate calls in half a second jam... we don't know the numbers involved. I theorize they're cheap as shit. Again, if this was a big deal, I'd understand. I'm just hypothesizing it is not. And I'm mainly angry that after 4 years of putzing about and wrecking the pre-burns of countless people, the best that BM was able to come up with (to perhaps cludge-fix the thing?) is this wave lottery idea. Uuughhhhh...

P.S. Also changing the tickets from 2-at-a-time to 1-at-a-time buy scheme is free. That was my suggestion #2.

[trilobyte]

Relax, Francis, a waiting room system is not only more fair than you think it's also more closely aligned to how all online ticketing has worked for the last several years (regardless of ticketing providers).

Ticket system opens gate.... if server capacity is greater than demand, then when the gate opens people get what they want, bar none,

If demand is greater than capacity, then failure occurs.... people potentially get tickets ahead of their place in a queue.

Doing a random chance waiting room system ensures not only the greatest possibility of getting tickets, but also the greatest shot at fairness.

Speaking personally, that's a real motherfucking crux. It's a chance to be able to get a ticket. Presales go to the wealthy, and directed group sales go to the groups who have demonstrated an ability to build and create great things on the playa.

Burning Man is making the play at using the waiting room system to ensure fairness to all those planning to arrive at the start of sales... versus some dodgy system that arbitrarily gives people an inside track based on <question mark> bullshit. I'm all for the people coming in early having equal access. No special privilege afforded to people with faster (or slower) internet connections... just make it all happen.

My thoughts, you're a jerk. No offense, I just don't think that reducing the purchase maximum (especially at this late stage) to a maximum of 1 is a great idea.

[tffy]

Haha, the funny thing is - I agree with pretty much it all! Not sure why I'm a jerk for illustrating options and ridiculing successive years of failure, but okay... facts stay on my side.

Raffle is fair, great. So, we went away from it in '13...why exactly? Oh, right, camp cores didn't get their share. Okay, well, pre-distribute 10k tix and then do a clean raffle! Instead they're doing this "waiting room" bullshit. You really think ANYONE who wants a ticket (even those who won't read the e-mail that came in at 6 F-ing PM on the day before the auction) will not be clicking at 12:00? So why all the complications? Apparently because clicking 'buy' in a 20-minute span of time is more 'egalitarian' than doing the same thing in a 1-second span. Just collect all the names and just do the draw then.

I am not a fan of inconsistency, amateur bullshit and hacked-together-solutions. The angst you're seeing is built up over the past 4 years of them f*cking this basic task up over and over and over... sparked by this 'last minute surprise.' The whole process is like watching someone try to break a pinata.

If raffles are the shit - stick to those. If raffles are the devil - don't stick to them. Instead of this whiffle-waffle-back-and-forth.

Oh, and 'sell the tickets in ones instead of twos' wasn't a suggestion to implement <checks watch> 11 hours before the sale. Don't know where you got that idea. Juuust a possible suggestion for the future. Kuz if we want egalitarian equality - each man/woman for him/herself is the way to go, haha!

[BBadger]

Hence the word "rent" that I used. Big boys like Amazon, who have spend a lot of money and smart-people-time on developing all the fast fancy high-power servers... they sell them, you know... and you don't gotta buy them for a whole year.

You can call up TicketFly and discuss the issues. They deal with some massive spikes and how they handle it is really their problem.

And anyway, if some burners are butthurt over the ticket process they can go boycott the event in protest.

With that in mind, projections about the 'incredible tons of untold internet dollars' needed to solve a once-a-year 100,000 separate calls in half a second jam... we don't know the numbers involved. I theorize they're cheap as shit. Again, if this was a big deal, I'd understand. I'm just hypothesizing it is not.

Yeah, yeah, the armchair I'm-not-a-network-and-database analyst speculating about costs and all that jazz... Realize that TicketFly is now responsible for managing the waiting room and queues and sales and all that crap. They already scale to meet demand, but sometimes the spikes can overwhelm these systems.

And I'm mainly angry that after 4 years of putzing about and wrecking the pre-burns of countless people, the best that BM was able to come up with (to perhaps cludge-fix the thing?) is this wave lottery idea. Uuughhhhh...

The fact of the matter is that this is a sale that takes place only once per year -- a very long feedback loop for refining the process. This is coupled with a lot of new outside problems such as sell-outs, scalpers, and other problems that are unknown territory for this event, and often don't provide timely warning. It's not like the BMOrg is running wargame simulations to predict outcomes. Nor do other events provide much guidance given the values this event seeks to preserve.

I am not a fan of inconsistency, amateur bullshit and hacked-together-solutions. The angst you're seeing is built up over the past 4 years of them f*cking this basic task up over and over and over... sparked by this 'last minute surprise.' The whole process is like watching someone try to break a pinata.

And yet here you are, the general after the war, on one hand whining about how things that "DID NOT WORK ONE BIT" in the past four years, and on the other complaining about trying something new. What is it you want again? All this whiffle-waffle-back-and-forth.

Will all 30,000 or places in the waiting room fill up in that 20 minute window making it a lottery? We'll find out. There were people here who couldn't even handle the task of registering for this event in the first place.

Is it any better or any worse than trying to time your click the moment the doors open? Who knows either. It's probably a wash either way. Maybe it'll just mean the server isn't so hammered.

In the end 30,000 tickets will be sold and life will go on, except for more "ticket solutions" which is always a constant in this brave new world.

[magpiejay]

I'm actually not opposed to the idea of a lottery, per se. I'd rather the round one waiting room access window was smaller, as twenty minutes is enough to get everyone in the waiting room anyway, but it seems fair enough - having been lucky and unlucky in previous years thanks to crashes etc.

If it was announced every year how it would play, then people would have registered multiple email addresses/burner accounts this time round. Changing it every year helps whack a few loophole moles.

I almost feel self reliance should come in the form of one date announcement, a few weeks ahead of time, and fewer email reminders. Though these days the internet is everyone's alarm clock.

Anyway, good luck all.

[trilobyte]

Trying to oversimplify how event ticketing works... on a really high demand on-sale, it is currently impossible for any company to create a truly fair first-come first-serve system. You can buy and rent all the server hardware you want, but so far no company (ticketmaster/livenation/etc) has been able to meet that kind of demand. When demand is significantly higher than the system's capacity, things are basically random. If the queues fail, you get what are called kickouts (when you're in the queue waiting, and then suddenly something fails and you have to start over), and you just get screwed over - have you ever tried to get tickets for a concert, been there right at the exact moment they go on sale and still not get through? That's largely because of random chance, which is what happens almost every single time a popular event goes on sale with the big ticketing companies.

Wall street transaction engines are different beasts in that there, the supply is virtually limitless - the potential variable is the price of the transaction. Sure, there are only a couple billion Facebook shares out there, but the way the stock market works is that at any given moment people are always and will always be selling. That's not how it works with tickets - some will change their minds eventually, but the vast majority today are only interested in buying.

Statistics about this year's main sale registrations has not been published, but we know that last year there were 80,000 registrants for the main sale. That is, 80K people wanted to buy 1 or 2 tickets. From past sales, each buyer gets an average 1.7 tickets. That means there was demand for 136,000 tickets. Demand for tickets this year is likely higher than that, and there are 10,000 fewer tickets available in the sale than there were in 2015.

I hated waiting rooms when I initially saw other sites using them. Partly because I was under the mistaken impression that first-come, first-served queues were flawless even under extreme demand, and partly because I was trying to buy tickets to events that were selling seats, and so every person that somehow managed to get ahead of me was going to be closer to the stage than I wanted to be. After some conversations with people at a company called AXS, as well as people who managed some on-sales and fan pre-sales for different bands I learned a lot and came around. Not only does a waiting room give the ticketing company a much better shot at successfully managing the flow, but it also gives them a much better opportunity to weed out bots and bad guys (scalpers using proxies, etc). I'm torn about how I feel about a 20 minute waiting room - on one hand that seems like a big window of time, on the other hand more time means doing a better job of weeding out the baddies.

[Jackass]

Uuuhhhh, yeaaaahh. Just like in '12.

Read up on people's overwhelming happiness with how THAT turned out.

Maybe BM is banking on well, we've distributed some "core group" tickets this time around so the guys/gals who are important won't raise the stink they did in '12. That's very likely what it is, actually. Maybe the stink will be a lot less.

Just funny that after 3 years of saying "Well that went baaaaad in '12" they (and on the last day before the raffle, mind you...) decided to give it a good-old-try one more time.

Um yeah, I was here in 2012. I don't have to read up on squat, I was a first responder. It's a lottery now, get used to it!

You, should definitely sign in at noon.

[ACfromSAC]

If I get a ticket, I will be a big fan of this year's lottery. If I don't, I will hate it. I have managed to get a ticket every year I wanted to attend, even in 2012 when I struck out on the lottery... so even if I miss out today, I will have faith that the ticket fairy will make it happen for me again.

[1durphul]

I predict that the waiting room thing is going to be a crazy clusterfuck. The BORG's first attempt at new things, especially technology ticket things, often results in much gnashing of teeth and pitchforked mobs as that new system inevitably crashes under the load.

So with that in mind, I wish you all good luck. Keep your cool. Play smart, and you might bring victory to your district. May the odds be ever in your favor.

And don't forget, you'll have undoubtedly countless opportunities during the year to buy from friends who realize they can't make it after all, or bought an extra ticket just in case somebody like you wasn't able to get one.

[vargaso]

Trying to oversimplify how event ticketing works... on a really high demand on-sale, it is currently impossible for any company to create a truly fair first-come first-serve system. You can buy and rent all the server hardware you want, but so far no company (ticketmaster/livenation/etc) has been able to meet that kind of demand. When demand is significantly higher than the system's capacity, things are basically random. If the queues fail, you get what are called kickouts (when you're in the queue waiting, and then suddenly something fails and you have to start over), and you just get screwed over - have you ever tried to get tickets for a concert, been there right at the exact moment they go on sale and still not get through? That's largely because of random chance, which is what happens almost every single time a popular event goes on sale with the big ticketing companies.

Wall street transaction engines are different beasts in that there, the supply is virtually limitless - the potential variable is the price of the transaction. Sure, there are only a couple billion Facebook shares out there, but the way the stock market works is that at any given moment people are always and will always be selling. That's not how it works with tickets - some will change their minds eventually, but the vast majority today are only interested in buying.

Statistics about this year's main sale registrations has not been published, but we know that last year there were 80,000 registrants for the main sale. That is, 80K people wanted to buy 1 or 2 tickets. From past sales, each buyer gets an average 1.7 tickets. That means there was demand for 136,000 tickets. Demand for tickets this year is likely higher than that, and there are 10,000 fewer tickets available in the sale than there were in 2015.

I hated waiting rooms when I initially saw other sites using them. Partly because I was under the mistaken impression that first-come, first-served queues were flawless even under extreme demand, and partly because I was trying to buy tickets to events that were selling seats, and so every person that somehow managed to get ahead of me was going to be closer to the stage than I wanted to be. After some conversations with people at a company called AXS, as well as people who managed some on-sales and fan pre-sales for different bands I learned a lot and came around. Not only does a waiting room give the ticketing company a much better shot at successfully managing the flow, but it also gives them a much better opportunity to weed out bots and bad guys (scalpers using proxies, etc). I'm torn about how I feel about a 20 minute waiting room - on one hand that seems like a big window of time, on the other hand more time means doing a better job of weeding out the baddies.

Logged on to say basically what Trilo says here. When tens of people log on to the server at literally the same millisecond, "first-come/first-served" becomes meaningless and the software queues users randomly anyway. So with the waiting room thing, the BMORG is acknowledging that fact and yeah, creating an instant lottery. A lot of people are not going to understand this and may feel like "oh great, another lottery." But due to high demand, we've had a de facto ticket lottery anyway, so might as well be up front about it.

[Jovankat]

Statistics about this year's main sale registrations has not been published, but we know that last year there were 80,000 registrants for the main sale. That is, 80K people wanted to buy 1 or 2 tickets. From past sales, each buyer gets an average 1.7 tickets. That means there was demand for 136,000 tickets. Demand for tickets this year is likely higher than that, and there are 10,000 fewer tickets available in the sale than there were in 2015.

We do know that there were about 70,000 registrations. And the average purchase last year was actually 1.87 tickets. So that suggests more like demand for 149,600 tickets in 2015 and if that purchase average holds 130,900 for this year.

So last year's to availability to demand ratio was like 1 to 3.74 and this year is probably more like 1 to 4.36.

Yeah math(s)!

[tamarakay]

tick tock tick tock

[trilobyte]

I've merged 1durphul's chaos thread with the existing storm thread.

[TT120]

I'm damn glad I'm not the one of the IT guys for Ticketfly today. Their servers are fixin to get blown the hell up.......

[1durphul]

I've merged 1durphul's chaos thread with the existing storm thread.

Thanks

[Luigi]

So I had a dream last night - that there was no cap on BM and I could just buy a ticket. Which made me think (when I woke up)

'how big would it be?'

Using trio's numbers about 150k? Or would it be just a little bigger than 75k? If every person could buy 1 non transferable ticket, and the sale was 24 hours, how many would buy one?

It was just a dream.....

[1durphul]

When demand is significantly higher than the system's capacity, things are basically random. If the queues fail, you get what are called kickouts (when you're in the queue waiting, and then suddenly something fails and you have to start over), and you just get screwed over - ...

This is essentially what happened to me in 2011, and why I wasn't able to go to the Burn that year. Boy was that upsetting! I spent hours in the queue and then it died, but then it let me back in, then it sold me a ticket, but it never charged my card. I had the receipt for purchase, but the BORG wouldn't honor it and just let me pay for the ticket.

[Jackass]

[Captain Goddammit]

So I had a dream last night - that there was no cap on BM and I could just buy a ticket. Which made me think (when I woke up)

'how big would it be?'

Using trio's numbers about 150k? Or would it be just a little bigger than 75k? If every person could buy 1 non transferable ticket, and the sale was 24 hours, how many would buy one?

It was just a dream.....

I've only been to BM twice since the capped sellout years... for over a decade I went and that dream of yours was the reality. Hell you could buy tickets at the gate. There used to be zero problem...

[gateway]

I'm damn glad I'm not the one of the IT guys for Ticketfly today. Their servers are fixin to get blown the hell up.......

Its not an easy situation to be in esp when demand really outweighs supply. I personally knew several people who started in the que at 11:45 ish and well where stuck, their are plenty of reasons why this would happen so I while a waiting room may seem fair from the outside what is actually happening under the hood. Did the array of users waiting get exhausted in memcache and start to drop off people at the end and pop on new id's to the beginning as the storm hit. Who knows..

btw, I was looking at an old ticket from last year , doesn't Ticket Fly charge $21ish service feed? If so and they sold 30k in tickets today they made about 630k (?) in about 1/2 hr.. so they better have their systems worked out and fair. Of course as a devops myself you can never foresee when issues at load can arise , blazemeter tests or not.

Im personally bummed I wasn't able to secure tickets this year.