About the Ravenports Universal Package System

Ravenports is an integrated system designed to build packages of complex software on all UNIX-like platforms. It is considered “universal” because Ravenports is not anchored to any single operating system; once the build procedure for a particular software is created (a.k.a “port”), that software is then available for all supported platforms. The simplest use case is to access the freely available prebuilt package repository for each supported platforms. More complex usage involves customizing builds to create private package repositories for specialized use. The product packages and repositories are manipulated with a package manager.

Notable advantages of Ravenports over other frameworks:

Inherent support of variant packages

Other frameworks may refer to these as “flavors” or “slave ports”. A variant package set is one that is built with a different configuration over the standard set. The use of variant packages eliminates the need for user-configurable options, meaning the consumer is likely to have access to a pre-built binary package and not have to build the software themselves.

Inherently higher quality build procedures

Most other frameworks are based on makefiles or shell scripts. Ravenports are compiled into a single “buildsheet”. Mistakes by developers are caught during the compilation process, so syntax and formatting are always correct. Unlike other frameworks, there’s exactly one way to build Ravenports, meaning that if a new port or modification builds successfully for a contributor, it will build identically for an evaluator. Submission checks are capable of being automated and performed against multiple platforms before a human ever sees them. This leds to much better submissions and eliminates unnecessary multiple review cycles.

Inherent support of subpackages

Other frameworks require separate ports if they want to split documentation, examples, libraries, or anything other components into separate packages to sanely satisfy dependency requirements. This requires redundant rebuilding and risks getting related ports out of sync. Ravenports allows many subpackages to be generated simultaneously by a single port and this capability is heavily used. This allows for fine-grain dependency specification (e.g. just pull a fortran runtime library instead of the entire GCC compiler collection.)

Multiple package formats possible

Currently only one package format is supported, which is the first version defined by pkg(8), the FreeBSD package manager. This package manager has been forked as ravensw(8) with new features as well as enhanced portability. It has been throughly tested on all platforms supported by Ravenports. Given interest, other package managers could be supported such as the Image Packaging System (IPS), Arch Linux’s pacman, Debian's DPKG family, etc.

Enterprise friendly

Ravenports was designed from the beginning to handle “custom” ports. These allow, among other things, to freeze software at a specific version. Companies are not forced to use the versions in quarterly sets or the bleeding-edge tree, nor is support tied to the End-of-Life (EOL) of any operating system release. Ports version tailoring can be provided as a service to the company which has the added benefit of receiving security updates on a weekly basis.

Using other frameworks can lead to frustration when help is urgently needed but all the developers contribute on a volunteer basis. Corporate users of Ravenports have the option of obtaining professional service when required.

Entire tree integrity

Other frameworks receive ports updates many times a day, sometimes numbering in the hundreds. Coupled with a much lower quality (a result of a wide range of volunteer developer skills coupled with little or no quality assurance checks and impact assessments), ports trees break frequently and often without addressing downstream breaking. Ravenports stages the changes for several days. A complete build of the ports tree is done to ensure tree integrity. The ports tree is published when ready after any fallout issues are addressed to avoid surprises. This also prevents temptation by users to update and rebuild daily, which would result in constant and frustrating build churn. Finally, a roughly weekly publishing schedule allows the ports tree and provided binary packages to remain in sync, making it possible to safely mix ports and packages.

Build performance

This advantage does not benefit system adminstrators that exclusively use prebuilt binary packages, but it is greatly appreciated by those that build their own packages. Ravenports can scan and process the ports tree hundreds to thousands of times faster than similar frameworks. Unlike the others, Ravenports has an administration/build utility that’s fully integrated and not a bolt-on tool. The utility, ravenadm, is written in Ada and capable of high concurrency to take full advantage of powerful build machines. Full tree scans which may take minutes (up to an hour on pkgsrc) are accomplished in less than 1 second by ravenadm. Every aspect of the framework is geared towards build quality and performance, resulting in significantly shorter build times over comparable frameworks.

Concurrently installed versions

Several popular projects maintain several branch releases. For example, the Perl project has 3 concurrent versions. Python, Ruby, most database servers and other projects do as well. Ravenports always supports two versions of Perl and provides packages for both which can be installed concurrently. The same case exists for Python for the last two 3.x releases.

The PHP branches have unique installations so even multiple versions of PHP can be installed together. Default database server versions can be specified per repository, but currently only a single version can be installed at a time.

Supported Platforms

Ravenports has been designed to support any POSIX-compliant operating system outside of Microsoft Windows. While it self-identifies as “universal”, Ravenports must be bootstrapped to the point of self-hosting on each supported architecture of each supported operating system. For the core developers, the main architectures targeted are the x86_64 and ARM64. New architecture support from serious contributors capable of performing the initial bootstrap and periodic quality-assurance builds of the entire Ravenports tree will be accepted into the project.

Current

Currently supported platforms

  • DragonFly 6.0 and later
  • FreeBSD/amd64 Release 12 and later
  • Linux/x86-64 (glibc 2.6.32-based)
  • macOS/Darwin x86_64 (on hold)
  • NetBSD/amd64
  • Solaris 10+ / illumos

Short Term

Platform support planned

  • FreeBSD/ARM64
  • Linux/ARM64
  • MidnightBSD/amd64

Long Term

Support increasingly unlikely

  • OpenBSD/amd64
  • i386 (Linux and *BSD)

A major requirement on platforms that are candidates for Ravenports support is that a recent version of GCC must be available on the platform, and must include fully functional C++, Fortran and Ada front-ends. For exotic platform/architecture combinations, this may require cross-compilation bootstrapping along with new support patches to bring Ada tasking capability to the candidate platform.

Quick Start

The section will be re-worked. The main User Guide has not been written. The quickstart guides linked below tell how to install the Ravenports Administration tool (ravenadm) with a host-specific system root and toolchain.

There are complete repositories for each of the supported platforms, so anyone familiar with pkg(8) on FreeBSD or DragonFly BSD will understand how to install the prebuilt binaries immediately. Linux users may require finding FreeBSD-based tutorials to grasp it better. This should be addressed in the upcoming user guides (TBW).

The ravenadm tool does have a complete set of man pages though. ravenadm help will show those pages. Those familiar with the (d)synth port building tool on DragonFly BSD or FreeBSD will quickly adapt to ravenadm, as it has the same look and feel, and also has the web-based dynamic build report.

Update Schedule

The source tree for Ravenports is under constant update. However, the Ravenports tree is only generated on roughly a weekly basis. Minor updates to heavily used ports can cause the majority of the tree to require a rebuild, so the policy is to provide all the updates in one step.

Perl ports

With many thousand perl modules with CPAN, updates are seen many times per day. Perl alone causes serious build churn based on these daily updates, and that’s without any kind of integrity checking. Therefore the policy is to update perl modules a couple of times per week. Before a release is tagged, a full build check is performed internally to ensure the perl ecosystem remains functional.

Python ports

Python modules are similar to perl, but are more volatile. Typically python modules are updated the same day as perl, and they are also integrity build-tested before a release is tagged.

Security and vulnerability updates

For non-commercial users, security updates appear as regular updates with every release. These updates may appear as version updates. Commercial service users that have specified and locked down software versions (older than the current Ravenports tree) will receive security as port revisions accompanied by patches each Tuesday. If patches aren’t feasible, the client will be informed as soon as possible to determine how best to address newly-known security issue.)

Binary package updates

As soon as the Ravenports tree is generated, incremental building of packages will commence on all supported repositories. Depending on the extent of the changes, new packages should be available within a few hours to a couple of days after the generation of the source tree. For commercial service clients, a notice will be sent when new packages are ready for download.

Volunteer Contributions

Every leg of the Ravenports Triad consists of Free and Open-Source Software (FOSS), released under the permissive Internet Systems Consortium (ISC) License. Contributions are welcome and may appear in many forms, such as:

  • bug reports
  • patch submissions
  • port adoption and maintenance
  • Binary package and repository generation
  • Security vulnerability reports
  • Submission of new ports

Bug and Security vulnerability reports

In the future, we may host a dedicated issue tracking system. For the time being, the Github issue tracking system is available for users to report bugs and any security issues that may not have been known or addressed yet.

Report here: https://github.com/ravenports/ravensource/issues

Submission of patches and new ports

In the future, submission of patches and new ports will be done through a dedicated interface that will put the submission in a build queue and test build the affected ports on all supported platforms automatically. If the build tests pass automated tested, the submissions will be forward for human inspection and inclusion. However, we aren’t there yet.

All patches must be build tested locally. If they pass the submitter’s own testing, the best approach is to submit the contribution as a github pull request here: https://github.com/ravenports/customsource/pulls

The process is the same when submitted an entirely new port, with one additional requirement. The person that creates and submits the port agrees to be listed as the port’s maintainer for at least 6 months. It is expected that submitter fully tests the new port on at least one platform.

The pull request description for a new port should provide the follow information:

  • Synopsis of what functionality the new port provides (what the software does)
  • Impact of not approving the port for inclusion
  • Optionally: what type of user is likely to benefit from its inclusion
  • Which platform(s) the port was tested on

Port adoption

Each port may have one or more maintainers. Most ports have no listed maintainer or are generated by a script. There are benefits to having a specific person or team of people responsible for maintenance and update of a port. Ideally the maintainer should be a regular user of the port’s software. This helps porting errors to be detected quickly and also allows for quick confirmation of reported bugs. A real user is more motivated to fix bugs and keep up with the latest software upgrades.

Those volunteering to be the primary maintainer of a port are expected to be familiar with the Ravenporter's Guide and have the technical skills necessary for basic port maintenance and troubleshooting. They are also expected to handle bug reports to their natural conclusion. If multiple maintainers are listed on a port, they are expected to communicate internally (each speak for the others on bug trackers, etc.).

Requests to adopt ports can be made either through the Github issue tracker or submitted as changes via a github pull request. The latter method is preferred.

Binary package and repository generation

This contribution forms when somebody wishes to extend Ravenports support to lesser-used platforms or architectures. The new platform support requires the contributor to have the necessary hardware to accomplish the system root and toolchain bootstrapping. Additionally the contributor is expected to build a complete set of binary packages and upload them at regular intervals. That will likely require per-platform adjustments to existing and incoming ports, so this is not a trivial contribution. Serious contributions are more than welcome, but there will be some vetting to ascertain the skill level and the commitment level of any person or team offering to extend Ravenports for non-commercial support use.

Latest News

2023-08-25 Major toolchain upgrade: All packages are now built with GCC 13.2.0 and binutils 2.41 instead of 11.2.0 and 2.35.1. Also the sysroots port has been split to one per operating system. DragonFly BSD and FreeBSD have had standard variants for versions 6.4 and 13.2 added in addition to the previous 6.2 and 12.2 respectively (rather then replacing them). Linux/glibc now has a new standard variant based on Debian 12 in addition to Ubuntu 16.04.
2023-04-17 MidnightBSD is now supported. Platform bring-up has been completed and an initial package set uploaded. A new USES module (mbsdfix) was added to the framework to automatically handle software with too old configure scripts.
2023-01-24 The packaging scheme is being reorganized: Development files will be split out into subpackages, same for manpages. It is expected to complete this for all the ports with single variants in two weeks or so. The rest will be converted over time during regular port version upgrades and maintenance. Also the first bunch of KDE ports have been committed. Work on that has stopped, though (missing wayland support on most platforms makes it impractical).
2022-10-15 A new Ravenports organization has been created on GitHub and all relevant repositories were transferred over.
2022-09-06 Ravenports switched away from GL libreries provided by mesa to those from libglvnd. While reworking many ports as part of the graphics stack migration, introspection has been made optional for many GTK-based ports.
2021-10-20 NetBSD support is now available. Various problems had to be solved to bring the modern Ravenports toolchain over and to work with some specifics of that platform. Work on fixing several ports is ongoing but a considerable number of packages are already available.
2021-08-25 Major toolchain upgrade: All packages are now built with GCC 11.2.0 and binutils 2.35.1 instead of 9.3.0 and 2.34. The exception here is Solaris since GCC 10+ do no longer support Solaris 10. The very old system libraries of 10u8 have become a more and more of a problem for various ports. The most likely path forward will be to drop Solaris support and pick a modern illumos distribution instead.
2020-12-26 FreeBSD base system has been upgraded from 11.1 to 12.2. This means that all packages built from now on require 12.2-RELEASE or later.
2020-06-07 Everything necessary to work with TeX has now been added to Ravenports, finishing a major porting effort. Packages for the full TeXlive 2020 distribution are available with subpackages for the engines and other components to pick from. The latest versions of the TeX IDE texstudio and the WYSIWYM TeX document processor LyX are available as well.
2020-05-11 Ravenports now supports CPE information. It has already been added to most of the existing ports.
2020-04-15 A new nls subpackage has been introduced and ports that previously supported internationalization via variants have been converted. It is expected that all new ports will offer localization support (if applicable) and the existing ones that could support it will do so in time, too. Also a port for the R language has been added as well as a facility to auto-create R ports from CRAN.
2019-11-22 Major toolchain upgrade: All packages are now built with GCC 9.2.0 and binutils 2.33.1 instead of 8.3.0 and 2.32.
2019-07-19 Ravenport's fork of FreeBSD's package manager has been renamed from pkg to ravensw and various additional portability fixes were added. Unfortunately it's still flaky on Linux.
2019-02-19 Ravenports supports installing more than one TLS library (libressl, openssl, libressl-devel, openssl-devel) on the same system at the same time. This makes it possible to install software with specific requirements that were mutually exclusive before. Support for the MacOS platform is on hold for now.
2018-12-22 Rust is now available on all platforms except Solaris/illumos and MacOS. With lumina (Qt-based) and Xfce4 (GTK+-based) two full desktop environments are available where basically all components have been ported. Alternatively there are various window managers to pick from.
2018-10-04 Initial support for MacOS High Sierra (10.13.6) and later has landed. This platform still requires some special workarounds. Therefore RP only provides pre-built packages but does not yet support custom package building.
2018-07-31 Major toolchain upgrade: All packages are now built with GCC 8.2.0 and binutils 2.31 instead of 7.3.0 and 2.30.
2018-05-25 Solaris 10u8 and newer as well as illumos are now supported (x86_64 only). However due to technical limitations, package building only works on 10u8 systems.
2018-04-03 Ports for almost all components of X11 are available for all supported platforms.
2017-10-05 Ravenports packages uses Zstandard compression instead of xz now. This speeds up the packaging phase considerably while yielding only slightly bigger packages.
2017-09-01 Website online. User guides still being developed. User feedback welcome.

Professional Services

Software Set Maintenance (source)

This service allows clients to specify the exact version or range of versions of software on a per-port basis they require. When the port is upgraded on the public Ravenports tree to a version newer than the client specifies, the client remains with the older specified version. Security sites are scanned daily for vulnerability announcements and patches for all ports used by the client. The client is sent a weekly report on all discovered vulnerabilities since the previous report. The client’s private Ravenports tree is updated every Tuesday. The client’s IT department uses the Ravenadm tool and the updated tree to build its own software packages.

Port creation service

If required software is not yet available in the Ravenports tree, it may be possible to pay to bring it in. The cost will vary depending on porting difficulty, and if it has been successfully ported to other package systems such as FreeBSD ports, pkgsrc, archlinux packages, etc. Keeping the new port limited to the client’s private ports tree (in other words, not placing the new port in the public Ravenports tree for others to enjoy) is possible but may affect the final cost. It’s recognized that legal requirements may require privacy, but companies are encouraged to contribute their new ports whenever possible.

Software Set Maintenance (source + binary packages)

This service maintains software to the client’s specifications as above, but also rebuilds the package repository. The client is sent a notification when the new packages and the changes that occurred over the previous week. This report is provided concurrently with the vulnerability briefing. The client’s IT department has the option to sync a local repository to the updated remote repository, or just directly update its machines from the remote repository.

Provision of new platform/architecture

A significant amount of time and work is required to bootstrap Ravenports on a new platform and/or architecture. Should a client find it a worthy investment, this support can be provided. The effort is a one-time cost; any required maintenance is considered standard work and not charged to a client.

Contact for Professional Services

Contact John Marino
email ravenports@ironwolf.systems
telephone +1 (479) 306-8850