Maintenance - 29-30th of September

Hi. Yes sorry, unplanned maintenance. Unfortunately this migration did not go as planned and is taking longer than we thought, I had a window to get it done this night/morning and jumped on it but that’s taking more time than I thought…

This maintenance should be that last one hopefully or at least we will warn you in advance if another intervention is needed. We will keep you updated here.

@rainerzufall we will update to nextcloud 14 soon, we usually give it a bit of time before majors upgrades, and getting this migration done is the priority right now :slight_smile:

1 « J'aime »

thanks for the quick response!

and good luck with the migration!

I would really appreciate a short update! I am sure you are doing your best, but information (i.e. a mail to all customers, but at least a forum post) is also important for the longevity of your service. Thanks!

1 « J'aime »

Hi, I can only pile up on the comment of @rainerzufall: keeping us informed is important to keep our trust.

cloud is back since yesterday

A bit more of background on the last events here https://forum.indiehosters.net/t/postmortem-cloud-migration/2915

@rainerzufall @jeremiejoudioux my apologies for the lack of information, there were a lot of unknown variables that I have to say lead to a bit of tensions :slight_smile:

We will improve this in the future, hopefully we should not get to such a critical situation anyways.

So far our main channel to reach all users is through nextcloud announcements but if its down that’s becoming an issue. The forum is probably the best way for such announcements.

thanks a lot for this update!

That’s the thing with communication. I was not sure whether it is really up, so I waited for your confirmation.
Interestingly I lost some of my one-time-passwords I have created for the Android Nextcloud app as well as Davdroid. Is that to be expected?

Happy to hear that as well!

unfortunately the lost of the password can be a side-effect

Thanks for the update and the more detailed insight linked above!
Do I understand it correcty that you transitioned all the data to the new infrastructure now? Or is there more to do?

Everything has been migrated to the new infrastructure, nothing more to do other than making sure everything is running fine there :slight_smile:

Hi there,
I am trying to update some scripts on my cloud and they get saved, but the updates don’t show on my webpage online…
Could you help me there?
Best,

Hi @mateo ,
we are going to fix this tomorrow see:
https://forum.indiehosters.net/t/maintenance-23rd-of-october-2018/2964

Ok, thanks, looking forward to it.
Best,
M

Are you still having troubles with your set up? Are you aware that over the last weeks at least once a day there are « Server Time Outs » and/or « Service Temporarily Unavailable »…

Is this already on your agenda also for the https://forum.indiehosters.net/t/maintenance-23rd-of-october-2018/2964?

@rainerzufall we are aware that there has been some up and downs but not that you get many timeouts.

Are you getting those timeouts when trying to login? Uploading big files or its random?

Yes, when trying to log in via a web browser. (but it should not be connected to my internet connection as everything else is working just fine.)

And my clients (desktop at work and also DAVdroid on my phone) do sometimes report connectivity issues saying that they can’t connect to my indie.host nextcloud (while my other two connected work nextclouds do work).

It is not connected to uploading (big) files.


If I can give more/better information, please just tell me how or what would be helpful.

Hi all

Just wanted to say that I also get very frequent issues since the maintenance, using the cloud apps through the browser (Firefox).

Sometimes, I get:

# 504 Gateway Time-out
nginx/1.13.12

Other times, I get:

# Not Found
The requested URL /apps/news/ was not found on this server.
Apache/2.4.25 (Debian) Server at cloud.indie.host Port 80

And other times, the page loads but I end up with only the top bar and nothing underneath.

me too… extremely slow for anything and sometimes time out.

@blub @stragu I know the cloud has been very slow lately. I’m still investigating on what is the bottleneck here… I will change a few things in the coming days to improve the performances on the php and mysql side, we will see if it gets better.

Plus I’d like to change the docker image we use, for the php-fpm one + nginx, but that will come a bit later.

Ultimately we will move the database to SSD disks but that comes with a cost that we need to think about.

A bit more background on what is going on:
So the cloud gets slower and slower as time goes on, at one point the health checks that we have setup restarts the cloud.

There was an issue, with the nextcloud apache config when the cloud restarts, leading to:

# Not Found
The requested URL /apps/news/ was not found on this server.
Apache/2.4.25 (Debian) Server at cloud.indie.host Port 80

Unfortunately I missed this a few times as it doesn’t appear in the monitoring, cloud restarts and you can login but you can’t get access to the pages.
This should not happen anymore.

Then when the cloud restarts, everyone is trying to reach the cloud in the same time, so yep timeouts.

Why is the cloud getting slower and slower, I’m still not sure. I changed a few things on the backup which could cause this issue, every 24h it is scanning and backuping millions of files.

2 « J'aime »

Sunday 04.11.2018:
I already noticed 6 times (14:09 / 15:22 / 21:18 / 21:32 / 21:47 / 22:00) a ‹ 502 Bad Gateway - The server returned an invalid or incomplete response. ›

Monday 5.11:
11:33
‹ 504 Gateway Time-out ›