UPDATE @ 9:04 PM PST: The search index has officially finished rebuilding and we are currently holding at 2,776,172 photos in the Aviation Photo Search Engine.

I'm pleased to say that everything is back to normal. Thank you to everyone for your patience and understanding during the past 24 hours! Time to get back to looking at awesome aviation photos again, just like this one:



Nice shot, Daniel!

---------

UPDATE @ 11:51 AM PST: Image servers have been brought back up, any broken images should be loading properly now. Our main search index has started repopulating and currently around 100k photos indexed. It will take several more hours for all 2.7 million photos to be index. Newly uploaded and viewed photos will appear in index immediately as they are accepted or their view counts are updated.

---------

UPDATE @ 10:58 AM PST: As many of you noticed, Airliners.net experienced a major outage starting yesterday afternoon and we are still not fully recovered as of this posting. The outage was caused by a power supply problem in our main data center in the United States. I still don't have all the details as to what happened, but we know that power to one of our main networks was completely severed while a technician was attempting to perform an upgrade.

This outage affected many sites besides Airliners.net, and our IT team has been up all night rebuilding servers for us.

As of this moment our database servers are all back online. In a few moments we still begin rebuilding our search databases, and this process will take up to 7 hours to complete, so you will see low numbers in search results until this is done. Also, our image servers are currently being rebuilt, so you may notice broken photos until this is completely resolved.

---------

Post will be updated with details soon.

Regards,

M

This should be an interesting discussion, seeing as I was under the impression that this sort of thing is exactly what the ill-handled site upgrade was supposed to prevent.

The photo search is not working since last night December 21, 2016 ,I am getting error 500

Search function is down. Trying to browse the last 24 hours photos from home page and get this error message:

Oops! An Error Occurred
The server returned a "500 Internal Server Error".
Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

I hope this crap will never happen again

It appears that the problem was not related to A.net's backend. Matt will tell you all what happened when he gets the chance.

Some functions (e.g. search) of the site aren't up and running yet, but they are being worked on.

As far as we know it was a local power outage, affected more than Anet so not related to the site upgrade

BTW why wasn't an update provided on this site's twitter or Facebook page during the outage ? It would have taken 30 seconds to write up a short explanation and would have kept the members informed of what was going on....

alberchico wrote:

BTW why wasn't an update provided on this site's twitter or Facebook page during the outage ? It would have taken 30 seconds to write up a short explanation and would have kept the members informed of what was going on....

Because those accounts are run by bots or employees that have no connection to the actual site itself. Social media "marketers," interns and the like.

The A.net social media account is mostly run by Paul (PanAm_DC10). He is our community manager and very much in touch with the site. He does a lot of work behind the scenes.

It would be good if you could be a bit more respectful to the crew and the work that we do.

alberchico wrote:

BTW why wasn't an update provided on this site's twitter or Facebook page during the outage ? It would have taken 30 seconds to write up a short explanation and would have kept the members informed of what was going on.

^ This.

Kept wondering why the Fbook and Twitter weren't updated, and could that indicate that the problem was more serious and all-encompassing than just an outage or update.

Really dropped the ball there, as that's something that should've definitely been done.

airkas1 wrote:

The A.net social media account is mostly run by Paul (PanAm_DC10). He is our community manager and very much in touch with the site. He does a lot of work behind the scenes.

It would be good if you could be a bit more respectful to the crew and the work that we do.

Why weren't there any updates or acknowledgement of an issue on any social media platform at all? That doesn't sound like they are very much in touch to me.

We were left in the dark yet again.

I mean whatever, it's a message board at the end of the day; doesn't mean we can't criticize it. Bring back the old site!

AMSATC wrote:

airkas1 wrote:

The A.net social media account is mostly run by Paul (PanAm_DC10). He is our community manager and very much in touch with the site. He does a lot of work behind the scenes.

It would be good if you could be a bit more respectful to the crew and the work that we do.

Why weren't there any updates or acknowledgement of an issue on any social media platform at all? That doesn't sound like they are very much in touch to me.

We were left in the dark yet again.

There was a post about it on the Facebook page, but I didn't see anything on Twitter.

9w748capt wrote:

I mean whatever, it's a message board at the end of the day; doesn't mean we can't criticize it. Bring back the old site!

Yes meanwhile lets bring back horse carriages to replace automobiles. We are not going back to the dark ages. The old forum worked as well as the bbs in the 1990s.

alberchico wrote:

BTW why wasn't an update provided on this site's twitter or Facebook page during the outage ? It would have taken 30 seconds to write up a short explanation and would have kept the members informed of what was going on....

If you take a look you would have seen a post was indeed made about the outage on our Facebook and Twitter accounts when we realised the severity of the problem. It was posted in a timely manner and asked for people's patience whilst the issues were being worked on. If it ever happens again, which I doubt, check our page and an update will be there, thank you.

Regards

LAX772LR wrote:

alberchico wrote:

BTW why wasn't an update provided on this site's twitter or Facebook page during the outage ? It would have taken 30 seconds to write up a short explanation and would have kept the members informed of what was going on.

^ This.

Kept wondering why the Fbook and Twitter weren't updated, and could that indicate that the problem was more serious and all-encompassing than just an outage or update.

Really dropped the ball there, as that's something that should've definitely been done.

You need to check your facts please, check our Facebook page and you'll find a post was made.

Regards

Time for critics to chill out. Doesn't matter if it is airliners.net, whitehouse.gov, Southwest, Delta, United...................... web sites are going to go down once in a while. If your addiction is too strong to bear a 24-48 loss of free service (or even paid service) you need to subscribe to internet.timeout.rehab.org

Merry Christmas
Gladelig Jul
Joyeux Noel
Mele Kalikimaka
Mo'adim Lesimkha
I'd Miilas Said Oua Sana Saida
Errymay Ristmaskay
Kala Christouyenna
Feliz Navidad

How often does this happen? Once every 15 years? Give em a break.

I checked Twitter specifically to see if there was an outage, but it was radio silence.

Now, I know the outage wasn't the fault of the guys who've put so much effort into rebuilding this site into a sustainable system that will work in the long term - they did their job, and they did it pretty well. Power issues at a data center can affect almost anyone at any time. It's just the luck of the draw. But there are some lessons to be learned here...

Number one, it seems to me that somebody needs to delegate additional access to more staffers for the social media accounts. Leaving it in the hands of one person, qualified or not (and he is - he's been a moderator almost as long as I've been a member if I remember correctly), can result sometimes in the broader population being left in the dark if things are accidentally overlooked. I'm certain the team was focused on the restoration of services.

One thing I will say, that many others won't: I'm a software engineer, and I understand the effort it takes to make things run as smoothly as you guys do. So, at that, I want to thank the team for all their hard work over the years. It's not easy - and you guys deserve some praise for what you've managed to do here (taking a homebuilt patched 5 million times over message board system, and converting it to a standardized system - while preserving the entire database of past messages - is no simple feat)... Great job, keep up the good work.

Chill guys. I went to the facebook page. Didn't see the post so I sent a message which was promptly and politely answered. Great to have it back up and working. More stable than some other sites I know.

Siren wrote:

Number one, it seems to me that somebody needs to delegate additional access to more staffers for the social media accounts. Leaving it in the hands of one person, qualified or not (and he is - he's been a moderator almost as long as I've been a member if I remember correctly), can result sometimes in the broader population being left in the dark if things are accidentally overlooked. I'm certain the team was focused on the restoration of services.

Hi there

There are 5 of us who run the social media it's just that I am the most active so the Leaf Group employees could have posted as well but I did it as they were so busy on fixing the site. As Tim noted we were online the whole time and we received many direct messages which were all answered within half an hour and those users were satisfied with the answers and feedback they received. We were on the ball on Facebook as that is the most active and we didn't receive too many messages on Twitter so we focussed on Facebook as we had the post up. If it ever happens again I encourage you all to check our Facebook page and an update will be there.

Thank you

Paul

AMSATC wrote:

airkas1 wrote:

The A.net social media account is mostly run by Paul (PanAm_DC10). He is our community manager and very much in touch with the site. He does a lot of work behind the scenes.

It would be good if you could be a bit more respectful to the crew and the work that we do.

Why weren't there any updates or acknowledgement of an issue on any social media platform at all? That doesn't sound like they are very much in touch to me.

We were left in the dark yet again.

Apparently, people didn't bother reading past the first Facebook post, which had been pinned to the top. I guess everyone assumed that something of this importance would have been the first post.

I actually read down and saw that there was indeed some word about the outage.

My own view is that the operation is run by some very-hard-working enthusiasts, most of whom volunteer their time. They can't be expected to think of everything the way some corporate site-manager would, because the A.net crew has experience with this site but not a zillion others; a corporate guy might have seen it before. I prefer the enthusiast crew that I have worked with for my decade-plus on a.net.

The bitching on here is also actually quite a compliment. People refer to themselves as "members", and are peeved because they are deprived of a.net for less than a day. Now THAT's loyalty! (Or obsession.) Regardless, in an odd way, it's something for the crew to be proud of.

Two a.net scenarios for this:

1) The DVD was involved
or
2) Let's blame the Russians!

Happy Holidays!

wjcandee wrote:

AMSATC wrote:

airkas1 wrote:

The A.net social media account is mostly run by Paul (PanAm_DC10). He is our community manager and very much in touch with the site. He does a lot of work behind the scenes.

It would be good if you could be a bit more respectful to the crew and the work that we do.

Why weren't there any updates or acknowledgement of an issue on any social media platform at all? That doesn't sound like they are very much in touch to me.

We were left in the dark yet again.

Apparently, people didn't bother reading past the first Facebook post, which had been pinned to the top. I guess everyone assumed that something of this importance would have been the first post.

I actually read down and saw that there was indeed some word about the outage.

My own view is that the operation is run by some very-hard-working enthusiasts, most of whom volunteer their time. They can't be expected to think of everything the way some corporate site-manager would, because the A.net crew has experience with this site but not a zillion others; a corporate guy might have seen it before. I prefer the enthusiast crew that I have worked with for my decade-plus on a.net.

The bitching on here is also actually quite a compliment. People refer to themselves as "members", and are peeved because they are deprived of a.net for less than a day. Now THAT's loyalty! (Or obsession.) Regardless, in an odd way, it's something for the crew to be proud of.

Unpaid volunteers doing all the forum moderation, all the photo screening, member interaction and social media advertising. Sounds perfect from the business aspect of the corporation running things.

Why can't I get people to work for free for me like this in the real world?

wjcandee wrote:

AMSATC wrote:

airkas1 wrote:

The A.net social media account is mostly run by Paul (PanAm_DC10). He is our community manager and very much in touch with the site. He does a lot of work behind the scenes.

It would be good if you could be a bit more respectful to the crew and the work that we do.

Why weren't there any updates or acknowledgement of an issue on any social media platform at all? That doesn't sound like they are very much in touch to me.

We were left in the dark yet again.

Apparently, people didn't bother reading past the first Facebook post, which had been pinned to the top. I guess everyone assumed that something of this importance would have been the first post.

I actually read down and saw that there was indeed some word about the outage.

My own view is that the operation is run by some very-hard-working enthusiasts, most of whom volunteer their time. They can't be expected to think of everything the way some corporate site-manager would, because the A.net crew has experience with this site but not a zillion others; a corporate guy might have seen it before. I prefer the enthusiast crew that I have worked with for my decade-plus on a.net.

The bitching on here is also actually quite a compliment. People refer to themselves as "members", and are peeved because they are deprived of a.net for less than a day. Now THAT's loyalty! (Or obsession.) Regardless, in an odd way, it's something for the crew to be proud of.

Excellent post, wjcandee. I'm glad the site was back up in less than 24 hours after a major power outage. Thanks to all who made it happen. I'm sure they didn't want to deal with a power outage on what is traditionally a short holiday week.

StTim wrote:

Chill guys. I went to the facebook page. Didn't see the post so I sent a message which was promptly and politely answered. Great to have it back up and working. More stable than some other sites I know.

Same for me.

When I woke (NZ time) I discovered the site was done. Concerned that it might be my system, I went to the Facebook and sent a message which was very promptly and very politely answered, and when I went back to check the Facebook page, I saw the announcement there.

mariner

It truly amazes me that people criticize and complain about some of the most petty things. If A.net went down, did it somehow stop your day? Did it hurt you in some sort of way? NO. It didn't. The fact that the crew is being as transparent as possible is very nice for those who were curious what happened. Sheesh. Get off their backs. 99% of the time nothing happens, and when the site hiccups, some of you come unglued. Just unreal.

Thanks for telling us! Hope you get stuff sorted out. I imagine how long this site has been around, it's a daunting task because of the size of everything. Best of luck!

Hi to Airliners.net admin and family, thank you for the update. I could imagine the amount of time and hard work you guys (ladies/girls) have put in! A hat off for all of you!

Merry Christmas and Happy New Year in advance from Malaysia! xD

You know, one of the reasons why I don't visit here as often as I used to is because there are SO many people on here who are never happy about anything this website does.

You folks must lead some incredibly unhappy lives if you have nothing better to do than to find any and every reason to criticize the management of this website while still coming back to this site on a regular basis.

If you're really that unhappy with the changes on this website and how the community is managed, then do us all a favor and leave.

Sure, it's frustrating when a website goes down unexpectedly and you don't know what's going on, but it's also important to remember that many of the folks managing the forums are VOLUNTEERS and live in different parts of the globe and may not even be awake during the same hours as you.

The attitudes of expectation and entitlement on here are astounding.

Please do yourselves a favor and go take a step outside. Take a deep breath of fresh air and let it out slowly. Regardless of which holidays you observe, it's the season. Enjoy it with your friends, families, and loved ones.

At the end of the day, getting bent out of shape over technical difficulties doesn't help anybody.

Cheers
- J

Thanks guys happy to be back online. Its an amazing job you do to create such an addictive website hence some of the comments.

Merry Xmas and Happy new Year to all A.NET addicts

JBo wrote:

Sure, it's frustrating when a website goes down unexpectedly and you don't know what's going on, but it's also important to remember that many of the folks managing the forums are VOLUNTEERS and live in different parts of the globe and may not even be awake during the same hours as you.

The attitudes of expectation and entitlement on here are astounding.

Please do yourselves a favor and go take a step outside.

At the end of the day, getting bent out of shape over technical difficulties doesn't help anybody.

Cheers
- J

Um, yeah, they were my thoughts too, and you have said it just right here.

Also, the site is free. It doesn't owe us anything. In that kind of sense, what is the expectation.. I missed the site, even for a day, but that's tech. It doesn't go down every other day, like the old one did moreso. I checked Facebook to confirm that there was a tech issue and that it would probably be up tomorrow. It was. I appreciate the work the volunteers are doing on the site day to day.

=

Some people just aren't happy unless they have something to be unhappy about

Thanks to all of the volunteer staff for all the time that they invest in keeping this site running regardless of how much moaning is directed at you.

wjcandee wrote:

The bitching on here is also actually quite a compliment. People refer to themselves as "members", and are peeved because they are deprived of a.net for less than a day. Now THAT's loyalty! (Or obsession.) Regardless, in an odd way, it's something for the crew to be proud of.

Hehe, indeed. Just like the good old days where A.net couldn't be down, otherwise people would be lost for the day From that point of view, you're absolutely right.
Also I don't mind criticism, as long as it's fact-based.

Happy holidays!

nice.I can now search for the photos again.Cool

Glad everything is back up and running and thanks for a wonderful site! Sure makes my work day go quickly.

It does though remind me when we lost connectivity to our data centre... It took 2 days to get the network back.. Transpires that someone drove a van round the back, security let them in by mistake and they stole the datacentre core router AND the backup!