Discussion:
RFC: Linux gABI: Add a GNU_PROPERTY_BY_LINKER property
H.J. Lu
2018-11-24 16:21:07 UTC
Permalink
Linkers group input note sections with the same name into one output
note section with the same name. One output note section is placed in
one PT_NOTE segment. Linkers with GNU property support merge all input
.note.gnu.property sections into one output .note.gnu.property section
with a single NT_GNU_PROPERTY_TYPE_0 note in a single PT_NOTE segment.
Since linkers without GNU property support treat input .note.gnu.property
section as a generic note section and just concatenate all input
.note.gnu.property sections into one output .note.gnu.property section
without merging them, we may see one or more NT_GNU_PROPERTY_TYPE_0 notes
in PT_NOTE segment, which is invalid.

GNU_PROPERTY_X86_UINT32_VALID was defined to address this issue such that
linker sets the bit in values of x86 properties for non-relocatable
outputs. But it isn't sufficient:

1. It doesn't cover generic properties.
2. When -mx86-used-note=yes is passed to x86 assembler, the
GNU_PROPERTY_X86_UINT32_VALID bit is set in GNU_PROPERTY_X86_ISA_1_USED
property in object file and linkers without GNU property support generate
invalid NT_GNU_PROPERTY_TYPE_0 notes with the GNU_PROPERTY_X86_UINT32_VALID
bit set.

I am proposing the following changes:

1. Add a GNU_PROPERTY_BY_LINKER property which should only be set by
linker for non-relocatable outputs to indicate the property note is
valid and generated by new linkers. Loaders can check this property
to verify that the property note is valid.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
3. Define GNU_PROPERTY_X86_ISA_1_BASE for GNU_PROPERTY_X86_ISA_1_USED,
which has the same bit as GNU_PROPERTY_X86_UINT32_VALID and use it
for -mx86-used-note=yes with x86 assembler.
--
H.J.
Florian Weimer
2018-11-26 13:12:33 UTC
Permalink
Post by H.J. Lu
GNU_PROPERTY_X86_UINT32_VALID was defined to address this issue such that
linker sets the bit in values of x86 properties for non-relocatable
1. It doesn't cover generic properties.
Okay.
Post by H.J. Lu
2. When -mx86-used-note=yes is passed to x86 assembler, the
GNU_PROPERTY_X86_UINT32_VALID bit is set in GNU_PROPERTY_X86_ISA_1_USED
property in object file and linkers without GNU property support generate
invalid NT_GNU_PROPERTY_TYPE_0 notes with the GNU_PROPERTY_X86_UINT32_VALID
bit set.
Surely this is a GAS bug? Why not fix that bug?
Post by H.J. Lu
1. Add a GNU_PROPERTY_BY_LINKER property which should only be set by
linker for non-relocatable outputs to indicate the property note is
valid and generated by new linkers. Loaders can check this property
to verify that the property note is valid.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
3. Define GNU_PROPERTY_X86_ISA_1_BASE for GNU_PROPERTY_X86_ISA_1_USED,
which has the same bit as GNU_PROPERTY_X86_UINT32_VALID and use it
for -mx86-used-note=yes with x86 assembler.
The alternative approach would be to switch to a new PT_ segment for
this because those aren't included in relocatable objects. (Maybe it's
time for another approach.)

Thanks,
Florian
H.J. Lu
2018-11-26 13:40:23 UTC
Permalink
Post by Florian Weimer
Post by H.J. Lu
GNU_PROPERTY_X86_UINT32_VALID was defined to address this issue such that
linker sets the bit in values of x86 properties for non-relocatable
1. It doesn't cover generic properties.
Okay.
Post by H.J. Lu
2. When -mx86-used-note=yes is passed to x86 assembler, the
GNU_PROPERTY_X86_UINT32_VALID bit is set in GNU_PROPERTY_X86_ISA_1_USED
property in object file and linkers without GNU property support generate
invalid NT_GNU_PROPERTY_TYPE_0 notes with the GNU_PROPERTY_X86_UINT32_VALID
bit set.
Surely this is a GAS bug? Why not fix that bug?
Linker removes GNU_PROPERTY_X86_ISA_1_USED when its value is empty.
Maybe linker shouldn't do that.
Post by Florian Weimer
Post by H.J. Lu
1. Add a GNU_PROPERTY_BY_LINKER property which should only be set by
linker for non-relocatable outputs to indicate the property note is
valid and generated by new linkers. Loaders can check this property
to verify that the property note is valid.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
3. Define GNU_PROPERTY_X86_ISA_1_BASE for GNU_PROPERTY_X86_ISA_1_USED,
which has the same bit as GNU_PROPERTY_X86_UINT32_VALID and use it
for -mx86-used-note=yes with x86 assembler.
The alternative approach would be to switch to a new PT_ segment for
this because those aren't included in relocatable objects. (Maybe it's
time for another approach.)
PT_NOTE is used so that binaries with GNU properties are backward
compatible with loaders which don't support GNU properties. They will
run without any new features from GNU properties.
--
H.J.
Cary Coutant
2018-11-26 22:33:50 UTC
Permalink
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
GNU_PROPERTY_X86_UINT32_VALID was defined to address this issue such that
linker sets the bit in values of x86 properties for non-relocatable
1. It doesn't cover generic properties.
Okay.
Does this imply that the property notes in all pre-existing binaries
can't be trusted?
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
2. When -mx86-used-note=yes is passed to x86 assembler, the
GNU_PROPERTY_X86_UINT32_VALID bit is set in GNU_PROPERTY_X86_ISA_1_USED
property in object file and linkers without GNU property support generate
invalid NT_GNU_PROPERTY_TYPE_0 notes with the GNU_PROPERTY_X86_UINT32_VALID
bit set.
Surely this is a GAS bug? Why not fix that bug?
Linker removes GNU_PROPERTY_X86_ISA_1_USED when its value is empty.
Maybe linker shouldn't do that.
Please explain how that answers Florian's question? You lost me.

What exactly are you saying the linker should not do? In your Aug. 10
proposal, ISA_1_USED is in the UINT32_OR_AND range, which specifically
says the bit should only be set in the output if *all* input files
contain the property (although it's unclear whether you meant "this
property is present in all relocatable input files" to mean a non-zero
property in all input files).
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
1. Add a GNU_PROPERTY_BY_LINKER property which should only be set by
linker for non-relocatable outputs to indicate the property note is
valid and generated by new linkers. Loaders can check this property
to verify that the property note is valid.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
3. Define GNU_PROPERTY_X86_ISA_1_BASE for GNU_PROPERTY_X86_ISA_1_USED,
which has the same bit as GNU_PROPERTY_X86_UINT32_VALID and use it
for -mx86-used-note=yes with x86 assembler.
The alternative approach would be to switch to a new PT_ segment for
this because those aren't included in relocatable objects. (Maybe it's
time for another approach.)
PT_NOTE is used so that binaries with GNU properties are backward
compatible with loaders which don't support GNU properties. They will
run without any new features from GNU properties.
With both your Aug. 10 proposal and this one, you're throwing
compatibility out the window by saying the loader shouldn't trust
those old notes without VALID bits. Can't we use this opportunity to
just do it right? At this point, I don't really care if you keep on
using SHT_NOTE for the properties in relocatable files, but please,
let's use a proper PT_GNU_PROPERTY segment for executables. (Sorry, I
promised to yield to the consensus, but the design keeps getting more
complicated.)

-cary
H.J. Lu
2018-11-27 13:40:36 UTC
Permalink
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
GNU_PROPERTY_X86_UINT32_VALID was defined to address this issue such that
linker sets the bit in values of x86 properties for non-relocatable
1. It doesn't cover generic properties.
Okay.
Does this imply that the property notes in all pre-existing binaries
can't be trusted?
Loader needs to make extra effort to check if property notes are valid:

https://sourceware.org/bugzilla/show_bug.cgi?id=23509
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
2. When -mx86-used-note=yes is passed to x86 assembler, the
GNU_PROPERTY_X86_UINT32_VALID bit is set in GNU_PROPERTY_X86_ISA_1_USED
property in object file and linkers without GNU property support generate
invalid NT_GNU_PROPERTY_TYPE_0 notes with the GNU_PROPERTY_X86_UINT32_VALID
bit set.
Surely this is a GAS bug? Why not fix that bug?
Linker removes GNU_PROPERTY_X86_ISA_1_USED when its value is empty.
Maybe linker shouldn't do that.
Please explain how that answers Florian's question? You lost me.
What exactly are you saying the linker should not do? In your Aug. 10
proposal, ISA_1_USED is in the UINT32_OR_AND range, which specifically
says the bit should only be set in the output if *all* input files
contain the property (although it's unclear whether you meant "this
property is present in all relocatable input files" to mean a non-zero
property in all input files).
No. A property can have zero bits. I updated my proposal to:

1. Add a GNU_PROPERTY_BY_LINKER property which should only be set by
linker for non-relocatable outputs to indicate the property note is
valid and generated by new linkers. Loaders can check this property
to verify that the property note is valid.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
1. Add a GNU_PROPERTY_BY_LINKER property which should only be set by
linker for non-relocatable outputs to indicate the property note is
valid and generated by new linkers. Loaders can check this property
to verify that the property note is valid.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
3. Define GNU_PROPERTY_X86_ISA_1_BASE for GNU_PROPERTY_X86_ISA_1_USED,
which has the same bit as GNU_PROPERTY_X86_UINT32_VALID and use it
for -mx86-used-note=yes with x86 assembler.
The alternative approach would be to switch to a new PT_ segment for
this because those aren't included in relocatable objects. (Maybe it's
time for another approach.)
PT_NOTE is used so that binaries with GNU properties are backward
compatible with loaders which don't support GNU properties. They will
run without any new features from GNU properties.
With both your Aug. 10 proposal and this one, you're throwing
compatibility out the window by saying the loader shouldn't trust
those old notes without VALID bits. Can't we use this opportunity to
This has been handled.
Post by Cary Coutant
just do it right? At this point, I don't really care if you keep on
using SHT_NOTE for the properties in relocatable files, but please,
let's use a proper PT_GNU_PROPERTY segment for executables. (Sorry, I
promised to yield to the consensus, but the design keeps getting more
complicated.)
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.

--
H.J.
Florian Weimer
2018-11-27 13:52:45 UTC
Permalink
Post by H.J. Lu
Post by Cary Coutant
just do it right? At this point, I don't really care if you keep on
using SHT_NOTE for the properties in relocatable files, but please,
let's use a proper PT_GNU_PROPERTY segment for executables. (Sorry, I
promised to yield to the consensus, but the design keeps getting more
complicated.)
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?

Thanks,
Florian
H.J. Lu
2018-11-27 14:00:38 UTC
Permalink
Post by Florian Weimer
Post by H.J. Lu
Post by Cary Coutant
just do it right? At this point, I don't really care if you keep on
using SHT_NOTE for the properties in relocatable files, but please,
let's use a proper PT_GNU_PROPERTY segment for executables. (Sorry, I
promised to yield to the consensus, but the design keeps getting more
complicated.)
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?
Kernel, glibc, gdb, .....
--
H.J.
Cary Coutant
2018-11-27 17:56:34 UTC
Permalink
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?
Kernel, glibc, gdb, .....
How is that any different from using PT_NOTE? Existing loaders don't
handle that either.

If you're claiming that a new PT_ value in the program header table
will be rejected by existing loaders, that's a bug. It should be
fixed. The whole principle of ELF extensibility relies on that.

-cary
H.J. Lu
2018-11-27 19:25:51 UTC
Permalink
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?
Kernel, glibc, gdb, .....
How is that any different from using PT_NOTE? Existing loaders don't
handle that either.
If you're claiming that a new PT_ value in the program header table
will be rejected by existing loaders, that's a bug. It should be
fixed. The whole principle of ELF extensibility relies on that.
Add PT_GNU_PROPERTY to cover .note.gnu.property section?
--
H.J.
H.J. Lu
2018-11-27 23:59:21 UTC
Permalink
Post by H.J. Lu
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?
Kernel, glibc, gdb, .....
How is that any different from using PT_NOTE? Existing loaders don't
handle that either.
If you're claiming that a new PT_ value in the program header table
will be rejected by existing loaders, that's a bug. It should be
fixed. The whole principle of ELF extensibility relies on that.
Add PT_GNU_PROPERTY to cover .note.gnu.property section?
Like:

1. Add PT_GNU_PROPERTY segment type.

# define PT_GNU_PROPERTY (PT_LOOS + 0x474e553)

which covers .note.gnu.property section.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
--
H.J.
H.J. Lu
2018-11-28 17:27:17 UTC
Permalink
Post by H.J. Lu
Post by H.J. Lu
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?
Kernel, glibc, gdb, .....
How is that any different from using PT_NOTE? Existing loaders don't
handle that either.
If you're claiming that a new PT_ value in the program header table
will be rejected by existing loaders, that's a bug. It should be
fixed. The whole principle of ELF extensibility relies on that.
Add PT_GNU_PROPERTY to cover .note.gnu.property section?
1. Add PT_GNU_PROPERTY segment type.
# define PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
which covers .note.gnu.property section.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
Here is a patch. The updated pdf is at

https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
--
H.J.
H.J. Lu
2018-12-04 13:57:45 UTC
Permalink
Post by H.J. Lu
Post by H.J. Lu
Post by H.J. Lu
Post by Cary Coutant
Post by H.J. Lu
Post by Florian Weimer
Post by H.J. Lu
PT_GNU_PROPERTY isn't compatible with existing loaders. This needs
to be both forward and backward compatible.
Which loaders? The kernel?
Kernel, glibc, gdb, .....
How is that any different from using PT_NOTE? Existing loaders don't
handle that either.
If you're claiming that a new PT_ value in the program header table
will be rejected by existing loaders, that's a bug. It should be
fixed. The whole principle of ELF extensibility relies on that.
Add PT_GNU_PROPERTY to cover .note.gnu.property section?
1. Add PT_GNU_PROPERTY segment type.
# define PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
which covers .note.gnu.property section.
2. Remove GNU_PROPERTY_X86_UINT32_VALID.
Here is a patch. The updated pdf is at
https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
Here is the patch to support PT_GNU_PROPERTY. Any comments?
--
H.J.
Loading...