Manjaro Difference between revisions of "Manjaro Mirrors"

Difference between revisions of "Manjaro Mirrors"

From Manjaro
imported>Fhdk
imported>Philm
Line 39: Line 39:
=What is the size of the Manjaro database?=
=What is the size of the Manjaro database?=


We are at around 50 GB. Minimum of 80 GB is recommended, since we might have bigger rebuilds, which might double our space need temporally. 100 GB or higher would be better. As an example see also [https://lists.manjaro.org/pipermail/manjaro-mirrors/Week-of-Mon-20151207/000013.html this].
We are at around 110 GB. Minimum of 150 GB is recommended, since we might have bigger rebuilds, which might double our space need temporally. 200 GB or higher would be better. As an example see also [https://lists.manjaro.org/pipermail/manjaro-mirrors/Week-of-Mon-20151207/000013.html this].


=How is the mirror synchronized?=
=How is the mirror synchronized?=
Line 133: Line 133:
=Current Size Required for a Manjaro Mirror=
=Current Size Required for a Manjaro Mirror=


''(last checked 2015-12-27 11:20 CET)''
''(last checked 2017-10-17 18:30 CET)''
<br clear="all" />
<br clear="all" />
1.6M ./testing/multilib/x86_64
  2.1M ./xperimental/multilib/x86_64
1.6M ./testing/multilib
  2.1M ./xperimental/multilib
540K ./testing/core/i686
  2.1M ./xperimental/core/i686
640K ./testing/core/x86_64
  2.4M ./xperimental/core/x86_64
1.2M ./testing/core
  4.4M ./xperimental/core
13M ./testing/extra/i686
  21M ./xperimental/extra/i686
15M ./testing/extra/x86_64
  23M ./xperimental/extra/x86_64
28M ./testing/extra
  44M ./xperimental/extra
19M ./testing/community/i686
  46M ./xperimental/community/i686
21M ./testing/community/x86_64
  49M ./xperimental/community/x86_64
39M ./testing/community
  95M ./xperimental/community
70M ./testing
  145M ./xperimental
1.6M ./stable/multilib/x86_64
  2.2M ./testing/multilib/x86_64
1.6M ./stable/multilib
  2.2M ./testing/multilib
540K ./stable/core/i686
  1.8M ./testing/core/i686
640K ./stable/core/x86_64
  2.0M ./testing/core/x86_64
1.2M ./stable/core
  3.7M ./testing/core
13M ./stable/extra/i686
  21M ./testing/extra/i686
15M ./stable/extra/x86_64
  22M ./testing/extra/x86_64
28M ./stable/extra
  43M ./testing/extra
19M ./stable/community/i686
  47M ./testing/community/i686
21M ./stable/community/x86_64
  51M ./testing/community/x86_64
39M ./stable/community
  97M ./testing/community
69M ./stable
  145M ./testing
1.6M ./unstable/multilib/x86_64
  2.2M ./stable/multilib/x86_64
1.6M ./unstable/multilib
  2.2M ./stable/multilib
540K ./unstable/core/i686
  1.8M ./stable/core/i686
640K ./unstable/core/x86_64
  2.0M ./stable/core/x86_64
1.2M ./unstable/core
  3.7M ./stable/core
14M ./unstable/extra/i686
  21M ./stable/extra/i686
15M ./unstable/extra/x86_64
  23M ./stable/extra/x86_64
28M ./unstable/extra
  43M ./stable/extra
19M ./unstable/community/i686
  46M ./stable/community/i686
21M ./unstable/community/x86_64
  51M ./stable/community/x86_64
39M ./unstable/community
  97M ./stable/community
70M ./unstable
  145M ./stable
44G ./pool/sync
  2.2M ./unstable/multilib/x86_64
4.2G ./pool/overlay
  2.2M ./unstable/multilib
48G ./pool
  1.8M ./unstable/core/i686
49G .
  2.0M ./unstable/core/x86_64
  3.7M ./unstable/core
  21M ./unstable/extra/i686
  23M ./unstable/extra/x86_64
  43M ./unstable/extra
  47M ./unstable/community/i686
  51M ./unstable/community/x86_64
  97M ./unstable/community
  145M ./unstable
  92G ./pool/sync
  8.7G ./pool/overlay
  100G ./pool
  101G .


=Check here before you send a request for mirror hosting=
=Check here before you send a request for mirror hosting=

Revision as of 16:31, 17 October 2017

What's a Mirror?

In the GNU/Linux distro world (& other systems too) a mirror is a server that hosts an up to date copy of a distro's software packages, stored in repositories (repos). There exist repos that are maintained by the distro administration - official - & other's. The Arch User Repository (AUR) being a good example of a non-official user maintained repo.

There are usually multiple repos in a mirror, holding software packages in categories, such as Manjaro's - core, extra, community & multilib repos. These repos will be duplicated with package content to suit both 32bit & 64bit installations, as well as for any & all the variety of releases that a distro may support. In Manjaro's case that is quite a number when all of the different Desktop & Window Manager titled front ends are considered.

The distro has package maintainers in its administration. They manage the contents of the repos, keeping it up to date, patching packages if required. Some distros, like Manjaro, have certain packages that are unique to it only - like mhwd for example.


Why do we need more mirrors?

The more mirrors we have the faster Manjaro's users can upgrade their systems. Some parts of the world have much faster internet speeds than others. In some circumstances, having a server in your country, or better yet, in your city, can make a world of difference to your download speeds.


How does Manjaro know what mirror(s) to use?

There is a file /etc/pacman.d/mirrorlist which lists all of the available mirrors.

There is another file called /etc/pacman-mirrors.conf that by default is configured to re-write your mirrorlist in a list with the fastest at the top, descending to the slowest, when the system is given the terminal command:

sudo pacman-mirrors -g && sudo pacman -Syy

However this command does NOT guarantee you have up-to-date mirrors.

The recommended and preferred way to ensure you get a mirrorlist which has up-to-date mirrors is to always use the -f{number} and remember to do pacman -Syy immediately after this command.

sudo pacman-mirrors -f10 && sudo pacman -Syy

This is a remarkably streamlined system compared to the ways that it has been in the past for the pacman rolling release system to manage its mirrors, let alone to rank their speed.

How do we get more mirrors?

I'm glad you asked that question. This is the advice of Philip Muller, Lead developer of Manjaro Linux:

'Some people still ask me how I got 5 mirrors in one week before 0.8.0 came out. It is simple. Just write 100 mails to universities and companies supporting Linux. Go to Arch linux and grab their mirrorlist. Crawl through their servers and get their contact data. Write an email and see what response you get. Here is some more info about it.'


What is the size of the Manjaro database?

We are at around 110 GB. Minimum of 150 GB is recommended, since we might have bigger rebuilds, which might double our space need temporally. 200 GB or higher would be better. As an example see also this.

How is the mirror synchronized?

Via the following Rsync-Services:


Asia / Japan:
rsync://ftp.tsukuba.wide.ad.jp/manjaro

Europe / Germany:
rsync://ftp.halifax.rwth-aachen.de/manjaro/

Europe / France:
rsync://fr.mirror.babylon.network/manjaro

Europe / Sweden:
rsync://ftp.lysator.liu.se/pub/manjaro/

Europe / Italy:
rsync://http://manjaro.mirror.garr.it/manjaro/

Europe / Netherlands:
rsync://nl.mirror.babylon.network/manjaro

Europe / United Kingdom:
rsync://rsync.mirrorservice.org/repo.manjaro.org/repos/

RU / Russian Federation:
rsync://mirror.yandex.ru/mirrors/manjaro/

CA / Canada:
rsync://ca.mirror.babylon.network/manjaro

It is always recommend to sync from the nearest location. To sync from our own manjaro.org server we need the IP of your server so it can be white-listed at our end. Please mail Philip Mueller directly. There is a recommended script for use with our server, so please contact the Manjaro administration in this regard.

How do you ask an organisation with servers to create a Manjaro mirror?

Write them an email like this (in your native language):


Dear Sir or Madam,

My name is [Your Name], I wish to ask you to please spare me a few minutes to read this letter & consider its contents.

I'd like to introduce you to a relative new comer to the world of Linux distributions - Manjaro Linux. Manjaro is a new user-friendly Linux distribution based on the highly regarded Arch Linux. We are currently searching for mirrors to host our packages.

Although new, we are already one of the top 10 most popular Linux Distributions in the world, as listed in the DistroWatch.com top 100.

Manjaro Linux is based on well tested snapshots of the Arch Linux repositories, and is 100% compatible with Arch itself. We manage our repositories with our own in-house tool called BoxIt, which is designed like git.

Our aim is to create a light Linux distribution which is simple, up to date, fast, user friendly and which follows the K.I.S.S (Keep It Simple, Stupid) principle. As such, Manjaro Linux provides a more user friendly installation process, utilities for managing graphic drivers, and pre-configured desktop environments.

Our repository is at the size of around 40 GB. We have between 50 GB and 100 GB granted on other mirrors hosting our packages. Currently it is possible to sync from the following rsync services:

Asia / Japan:
rsync://ftp.tsukuba.wide.ad.jp/manjaro

Europe / Germany:
rsync://ftp.halifax.rwth-aachen.de/manjaro/

Europe / France:
rsync://fr.mirror.babylon.network/manjaro

Europe / Sweden:
rsync://ftp.lysator.liu.se/pub/manjaro/

Europe / Italy:
rsync://http://manjaro.mirror.garr.it/manjaro/

Europe / Netherlands:
rsync://nl.mirror.babylon.network/manjaro

Europe / United Kingdom:
rsync://rsync.mirrorservice.org/repo.manjaro.org/repos/

RU / Russian Federation:
rsync://mirror.yandex.ru/mirrors/manjaro/

CA / Canada:
rsync://ca.mirror.babylon.network/manjaro

If possible, please sync from the nearest rsync-service to your location. Also, it is recommended to use a similar script as that we provide for our Tier1-Servers, to sync from our server:

https://github.com/manjaro/boxit/blob/master/manjaroreposync

Thank you for your time and consideration,

[Your Name]

Current Size Required for a Manjaro Mirror

(last checked 2017-10-17 18:30 CET)

 2.1M	./xperimental/multilib/x86_64
 2.1M	./xperimental/multilib
 2.1M	./xperimental/core/i686
 2.4M	./xperimental/core/x86_64
 4.4M	./xperimental/core
 21M	./xperimental/extra/i686
 23M	./xperimental/extra/x86_64
 44M	./xperimental/extra
 46M	./xperimental/community/i686
 49M	./xperimental/community/x86_64
 95M	./xperimental/community
 145M	./xperimental
 2.2M	./testing/multilib/x86_64
 2.2M	./testing/multilib
 1.8M	./testing/core/i686
 2.0M	./testing/core/x86_64
 3.7M	./testing/core
 21M	./testing/extra/i686
 22M	./testing/extra/x86_64
 43M	./testing/extra
 47M	./testing/community/i686
 51M	./testing/community/x86_64
 97M	./testing/community
 145M	./testing
 2.2M	./stable/multilib/x86_64
 2.2M	./stable/multilib
 1.8M	./stable/core/i686
 2.0M	./stable/core/x86_64
 3.7M	./stable/core
 21M	./stable/extra/i686
 23M	./stable/extra/x86_64
 43M	./stable/extra
 46M	./stable/community/i686
 51M	./stable/community/x86_64
 97M	./stable/community
 145M	./stable
 2.2M	./unstable/multilib/x86_64
 2.2M	./unstable/multilib
 1.8M	./unstable/core/i686
 2.0M	./unstable/core/x86_64
 3.7M	./unstable/core
 21M	./unstable/extra/i686
 23M	./unstable/extra/x86_64
 43M	./unstable/extra
 47M	./unstable/community/i686
 51M	./unstable/community/x86_64
 97M	./unstable/community
 145M	./unstable
 92G	./pool/sync
 8.7G	./pool/overlay
 100G	./pool
 101G	.

Check here before you send a request for mirror hosting

Following is a list that will be updated when required.

It is very important that we don't send requests to anyone on this list, as they will consider it spam & could have Manjaro's IP addresses blocked by organisations who's business is to attempt to control spam.

So when you are trying to organise new mirrors DO NOT bother hosters who have already denied us support:


23-05-2013
Canada
----------------------
csclub.uwaterloo.ca  - We are not currently
interested in mirroring Manjaro Linux.


It is a good idea to check repo.manjaro.org[1] to be sure that the wiki is up to date & also if you have been denied a request to post it in the same thread for obvious reasons.

Do You Manage a Potential Manjaro Mirror Server?

If so, here are some guidelines for you to think about, as putting up such a Mirror requires a certain commitment & perseverance.

The Manjaro community is better off not having a mirror, if it isn't kept functioning & up to date. So if you aren't in it for the long run, don't do it. You will only create disappointment.

Do you have enough bandwidth for the job? Do you pay for traffic? Your traffic may increase a lot. If you don't have enough bandwidth, you will end up offering at best a slow mirror, at worst an unreachable mirror.

Do you have enough disk space? You will need 50GB, though being able to offer 100GB would be great for the future. If your drive fills up you will not be able to rsync your mirror, it will become out of date & cause problems.

Sync every six hours. Being a rolling release system Manjaro's repos are very dynamic. So mirrors need to be updated multiple times per day.

Keep an eye on your sync scripts. Make sure that your mirror updates are functioning correctly. Users depend on your data to be all there & current.

Watch the Manjaro forum for announcements re. changes that may effect the mirror/repo system. The Manjaro administration will send you an email for any important changes. Though it often helps to be primed & ready for a change.

Cookies help us deliver our services. By using our services, you agree to our use of cookies.