Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 934514

Summary: [Future EAPI] Drop rewriting of absolute symbolic links (section 13.4)
Product: Gentoo Hosted Projects Reporter: Ulrich Müller <ulm>
Component: PMS/EAPIAssignee: Package Manager Specification <pms>
Status: IN_PROGRESS ---    
Severity: normal CC: dev-portage, esigra, flow, pacho
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=416431
Whiteboard: in-eapi-9
Package list:
Runtime testing required: ---
Bug Depends on: 933499    
Bug Blocks: 174380    
Attachments: foo-1.ebuild for package app-misc/foo in local overlay

Description Ulrich Müller gentoo-dev 2024-06-18 14:27:53 UTC 
PMS specifies in section 13.4.1 "Rewriting":
"Any absolute symlink whose link starts with D must be rewritten with the leading D removed. The package manager should issue a notice when doing this."
https://projects.gentoo.org/pms/8/pms.html#x1-14800013.4.1

AFAICS, Portage has implemented this in two places, function mergeme (in dbapi/vartree.py) and function movefile (in util/movefile.py).

The problem is that upon merging, D can have a value different from the value it had in src_* phases. (Again, this is specified by PMS: https://projects.gentoo.org/pms/8/pms.html#x1-10900011.1)

Especially, when a binary package is merged on a target host with PORTAGE_TMPDIR different from the build host's PORTAGE_TMPDIR, symlink rewriting will fail.

I am going to attach a MWE and a log that demonstrates the problem.
Comment 1 Ulrich Müller gentoo-dev 2024-06-18 14:28:54 UTC 
Created attachment 896026 [details]
foo-1.ebuild for package app-misc/foo in local overlay
Comment 2 Ulrich Müller gentoo-dev 2024-06-18 14:33:22 UTC Comment hidden (obsolete)
Comment 3 Ulrich Müller gentoo-dev 2024-06-18 14:38:29 UTC 
The following log demonstrates the problem.

Build and install the package on the build host (with PORTAGE_TMPDIR=/var/tmp):

   # emerge -1 --buildpkg app-misc/foo
   Calculating dependencies... done!
   Dependency resolution took 1.66 s (backtrack: 0/20).


   >>> Verifying ebuild manifests

   >>> Emerging (1 of 1) app-misc/foo-1::local
   >>> Unpacking source...
   >>> Source unpacked in /var/tmp/portage/app-misc/foo-1/work
   >>> Preparing source in /var/tmp/portage/app-misc/foo-1/work ...
   >>> Source prepared.
   >>> Configuring source in /var/tmp/portage/app-misc/foo-1/work ...
   >>> Source configured.
   >>> Compiling source in /var/tmp/portage/app-misc/foo-1/work ...
   >>> Source compiled.
   >>> Test phase [not enabled]: app-misc/foo-1

   >>> Install app-misc/foo-1 into /var/tmp/portage/app-misc/foo-1/image
   >>> Completed installing app-misc/foo-1 into /var/tmp/portage/app-misc/foo-1/image

    * Final size of build directory: 0 KiB
    * Final size of installed tree:  1 KiB

   >>> Done.

   >>> Installing (1 of 1) app-misc/foo-1::local

   >>> Completed (1 of 1) app-misc/foo-1::local

    * GNU info directory index is up-to-date.

Package installed on the build host, rewriting of the symlink has worked:

   # ls -l /usr/share/foo
   total 1
   lrwxrwxrwx 1 root root 21 Jun 18 16:11 link -> /usr/share/foo/target
   -rw-r--r-- 1 root root  5 Jun 18 16:11 target

Install from the binary package, with different PORTAGE_TMPDIR:

   # PORTAGE_TMPDIR=/tmp emerge -1 --usepkgonly app-misc/foo
   Calculating dependencies... done!
   Dependency resolution took 1.62 s (backtrack: 0/20).


   >>> Emerging binary (1 of 1) app-misc/foo-1::local
    * foo-1-2.gpkg.tar MD5 SHA1 size ;-) ...                                [ ok ]
   >>> Extracting info
   >>> Extracting app-misc/foo-1

   >>> Installing (1 of 1) app-misc/foo-1::local
    * QA Notice: Symbolic link /usr/share/foo/link points to /var/tmp/portage/app-misc/foo-1/image/usr/share/foo/target which does not exist.

   >>> Completed (1 of 1) app-misc/foo-1::local

    * Messages for package app-misc/foo-1:

    * QA Notice: Symbolic link /usr/share/foo/link points to /var/tmp/portage/app-misc/foo-1/image/usr/share/foo/target which does not exist.

    * GNU info directory index is up-to-date.

Rewriting of the symlink failed:

   # ls -l /usr/share/foo
   total 1
   lrwxrwxrwx 1 root root 58 Jun 18 16:12 link -> /var/tmp/portage/app-misc/foo-1/image/usr/share/foo/target
   -rw-r--r-- 1 root root  5 Jun 18 16:11 target
Comment 4 Mike Gilbert gentoo-dev 2024-06-18 14:46:29 UTC 
It seems to me that the rewriting should take place just after the src_install phase, before the binpkg is created. Would PMS allow for this?
Comment 5 Ulrich Müller gentoo-dev 2024-06-18 18:16:58 UTC 
(In reply to Mike Gilbert from comment #4)
> It seems to me that the rewriting should take place just after the
> src_install phase, before the binpkg is created. Would PMS allow for this?

Yes, I think so.

Portage should also implement the second part of what is specified: "The package manager should issue a notice when doing this." So we can get an idea how often such rewriting occurs.
Comment 6 Ulrich Müller gentoo-dev 2024-06-19 18:00:56 UTC 
A scan of binary packages showed no symlinks pointing to Portage's image directory, so looks like rewriting is not needed.

Suggestion: Drop rewriting of symbolic links in the next EAPI. Any bad symlinks resulting from this should trigger a QA warning (already with the current code). These can then be reported upstream, and the ebuild can work around the problem in src_install.

Reassigning to PMS.
Comment 7 Larry the Git Cow gentoo-dev 2024-06-27 06:12:39 UTC 
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=4d17764863896903df4d18fe7c6b45635a18f6c4

commit 4d17764863896903df4d18fe7c6b45635a18f6c4
Author:     Ulrich Müller <ulm@gentoo.org>
AuthorDate: 2024-06-19 08:08:04 +0000
Commit:     Ulrich Müller <ulm@gentoo.org>
CommitDate: 2024-06-19 18:17:06 +0000

    vartree, movefile: Warn when rewriting a symlink
    
    See PMS section 13.4.1 (Rewriting):
    Any absolute symlink whose link starts with D must be rewritten with
    the leading D removed. The package manager should issue a notice when
    doing this.
    
    Bug: https://bugs.gentoo.org/934514
    Signed-off-by: Ulrich Müller <ulm@gentoo.org>

 NEWS                         |  2 ++
 lib/portage/dbapi/vartree.py | 10 ++++++++++
 lib/portage/util/movefile.py |  5 +++++
 3 files changed, 17 insertions(+)
Comment 8 Larry the Git Cow gentoo-dev 2024-09-11 01:30:51 UTC 
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=02d0e00a1ba811b39140d10e17488f7fc3916534

commit 02d0e00a1ba811b39140d10e17488f7fc3916534
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-09-11 01:30:10 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-09-11 01:30:30 +0000

    sys-apps/portage: add 3.0.66
    
    Closes: https://bugs.gentoo.org/435066
    Closes: https://bugs.gentoo.org/907061
    Closes: https://bugs.gentoo.org/910560
    Closes: https://bugs.gentoo.org/933433
    Closes: https://bugs.gentoo.org/934220
    Closes: https://bugs.gentoo.org/934514
    Closes: https://bugs.gentoo.org/934784
    Closes: https://bugs.gentoo.org/935830
    Closes: https://bugs.gentoo.org/936273
    Closes: https://bugs.gentoo.org/937384
    Closes: https://bugs.gentoo.org/937485
    Closes: https://bugs.gentoo.org/937740
    Closes: https://bugs.gentoo.org/937888
    Closes: https://bugs.gentoo.org/937891
    Closes: https://bugs.gentoo.org/938127
    Closes: https://bugs.gentoo.org/933499
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.66.ebuild | 227 +++++++++++++++++++++++++++++++++
 2 files changed, 228 insertions(+)