[ih] fragmentation (Re: Could it have been different? [was Re: vm vs. memory])

[Paul Vixie]

Brian E Carpenter wrote:
...

Now that IPv4 is truly hitting its limits, the main operational complaint
against IPv6 is that it's too different from IPv4. But the incentives are
finally shifting in IPv6's favour, like it or not.

i don't even know what i don't like anymore. but for the history books
that may be written about our era, if indeed we have a future at all:

tony li said that ipv6 was too little, too soon. this was a play on
words, because the usual complaint is "too little, too late". tony was
right, even moreso than i realized at the time. we specified a lot of
things that didn't work and had to be revised or thrown out -- because
we did not know what we needed and we thought we were in a hurry. we had
time, as history will show, to spend another ten years thinking about ipng.

where we are today is that fragmentation is completely hosed. pmtud does
not work in practice, and also cannot work in theory due to scale
(forget speed-- it's scale that kills.) the only reliable way to
communicate with ipv6 is to use a low enough MTU that it never exceeds
any link MTU. in practice that means an ipv6 payload, plus its headers,
has to fit in an ethernet packet, so, 1500 octets. you can special case
the on-link LAN scenario so that if you have 9000 octets available you
can use them -- but that's the only time you can use more than about
1200 octets for your payload.

this means one of ipv6's major claimed-up-front advantages, which is
that only endpoints will fragment rather than gateways doing so as in
ipv4, never came about. in fact, ipv6 is far worse than ipv4 in this
way, as we learned by using ip fragmentation on UDP/53 (my idea: bad!)

this also means that we're chained to the MTU of the second-generation
10-megabit ethernet, which was carefully sized to fit a bunch of radio
spectrum and cable length parameters which have never applied since
then. but the IEEE 802 people know they're stuck with 1500 forever,
since no next generation of ethernet can succeed without being able to
transparently bridge onto the previous generation.

history is hard, let's do math.

; (155*10^6) / (53*8)
~365566.03773584905660377358
; (10*10^6) / (1500*8)
~833.33333333333333333333
; (100*10^6) / (1500*8)
~8333.33333333333333333333
; (1000*10^6) / (1500*8)
~83333.33333333333333333333
; (10000*10^6) / (1500*8)
~833333.33333333333333333333
; (40000*10^6) / (1500*8)
~3333333.33333333333333333333
; (100000*10^6) / (1500*8)
~8333333.33333333333333333333

right, so ATM failed in the market for a lot of reasons (state is what
kills, not speed, like i said) but one of those reasons was that an OC3C
at line rate was carrying too many cells per second to be able to handle
all of their headers in then-current or even projected-soon electronics.
we were wrong, and ATM has been used at OC12C, OC48C, and i've even seen
OC192C and OC768C, truly a testament to human cussedness fit for a
bumper sticker or perhaps a t-shirt.

looks to me like less than half a 10GBE is just as bad, and that at
40GBE and 100GBE it's well beyond absurdity. thankful as we are for
moore's law, i regret like anything the inability to send large enough
packets in the WAN so that we don't all need a 100 kilowatt routers to
handle the headers.

ipv6's mindless and unnecessary early adoption of an unworkable
fragmentation regime has chained my applications and those of my
children and their children to the maximum size of a packet in a closed
10-megahertz radio network. yay us.

[Joe Touch]

Post by Paul Vixie
where we are today is that fragmentation is completely hosed. pmtud does
not work in practice,

I do agree there are problems with fragmentation, but some layer of
fragmentation/reassembly is required when any layer below establishes a
"maximum" on message sizes, otherwise no amount of data origin guessing
will overcome a given level of layering and encapsulation.

Especially with PMTUD, but I thought that's why we've shifted to PLPMTUD.

IP fragmentation isn't working as intended, IMO because most routers
don't stay in their lanes - by forwarding only on IP header info.
Devices that peek into contents (DPI) necessarily need to reassemble
(even if as a shadow copy); claiming that "reassembly doesn't belong in
routers" while doing DPI is cherry-picking which parts of the router
requirements to follow and which to break.

(yes, I know PLPMTUD doesn't currently work with UDP, which is why I've
been working on extensions to UDP to support transport-layer fragmentation)

Joe

[Dave Crocker]

Post by Paul Vixie
ony li said that ipv6 was too little, too soon. this was a play on
words, because the usual complaint is "too little, too late". tony was
right, even moreso than i realized at the time. we specified a lot of
things that didn't work and had to be revised or thrown out -- because
we did not know what we needed and we thought we were in a hurry.

What you've actually summarized falls under 'too /much/ too soon'. The
'too much' part matches what I believe I saw.

However rather than 'too soon', one of the problems with the effort was
that in fact people made a point of not being sufficiently in a hurry.
This included explicit IETF presentations -- complete with charts --
purporting to prove that we had a decade (or whatever) before we
actually had to make the change.

So the actual practice of the effort is, IMO, better summarized as 'too
much too late' (and with no serious attention to adoption incentives or
adoption effort.

The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.

This was mere scope creep. Whether because of second system syndrome or
a failure to sufficiently feel the urgency of getting something fielded
and working sooner rather than later, I don't know. But I suspect it
was both.

d/

[Paul Vixie]

...
The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.
...

that word, "mandate," i don't think it means what you think it means.

we were volunteers, who worked on what we found deserving of our time.
or we were employees, who worked on what our bosses said to work on. at
no time did we or our families or our employers sign a suicide pact
agreeing to spend the next 16 years of our lives working on a system
whose utility could only be judged in the future, but which looked
pretty awful in the moment.

if the fragmentation differences between v4 and v6 were as you say a
form of scope creep, then i call foul, not on those engineers, but on
the paper pushing bureaucrats who killed TCPv6, which would have given
us renumberable connection endpoints. that is, if one was allowed, the
other should also have been.

i was pushing for a simple expansion of the IP header so that we could
use source routing on all flows, to connect network 10 at each end,
through a series of tubes, really, that had unique IP addresses, so that
the path would become the identity. the dns portion of this design
looked a lot like what was later called 8+8. i was shouted down, as was
mike o'dell, so i harken to your suspicion that anything simple would've
been rejected.

[Brian E Carpenter]

Post by Paul Vixie

...
The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.
...

that word, "mandate," i don't think it means what you think it means.
we were volunteers, who worked on what we found deserving of our time.
or we were employees, who worked on what our bosses said to work on. at
no time did we or our families or our employers sign a suicide pact
agreeing to spend the next 16 years of our lives working on a system
whose utility could only be judged in the future, but which looked
pretty awful in the moment.
if the fragmentation differences between v4 and v6 were as you say a
form of scope creep, then i call foul, not on those engineers, but on
the paper pushing bureaucrats who killed TCPv6, which would have given
us renumberable connection endpoints. that is, if one was allowed, the
other should also have been.
i was pushing for a simple expansion of the IP header so that we could
use source routing on all flows, to connect network 10 at each end,
through a series of tubes, really, that had unique IP addresses, so that
the path would become the identity. the dns portion of this design
looked a lot like what was later called 8+8. i was shouted down, as was
mike o'dell, so i harken to your suspicion that anything simple would've
been rejected.

Did you take it as far as a BOF?

https://www.cs.auckland.ac.nz/~brian/draft-carpenter-aeiou-00.txt
went to a BOF but was politely stomped on (IETF 29, Seattle).

Brian

[Paul Vixie]

Post by Brian E Carpenter

Post by Paul Vixie
i was pushing for a simple expansion of the IP header so that we could
use source routing on all flows, to connect network 10 at each end,
through a series of tubes, really, that had unique IP addresses, so that
the path would become the identity. the dns portion of this design
looked a lot like what was later called 8+8. i was shouted down, as was
mike o'dell, so i harken to your suspicion that anything simple would've
been rejected.

Did you take it as far as a BOF?

no. at that time i hadn't attended any IETF meetings and i didn't know
anybody who did, except my colleagues at DECWRL.

Post by Brian E Carpenter
https://www.cs.auckland.ac.nz/~brian/draft-carpenter-aeiou-00.txt
went to a BOF but was politely stomped on (IETF 29, Seattle).

your design was better considered than mine, and in my opinion would
have worked sooner and better than we're seeing from IPv6. you should
not have patterned it as a stopgap, that's probably what doomed it.

[Dave Crocker]

So, I see that I've touched a nerve. I'd expected to touch one, but not
this one. I'll try to respond carefully; please be gentle, as I fail...

Post by Paul Vixie

The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.

that word, "mandate," i don't think it means what you think it means.

Of course I do, and it's purely luck that Craig made the point for me:
cf, RFC 1726. That was not produced quickly nor in isolation nor by
only a tiny collection of wayward folk. It creates a mandate to work on
a particular problem to a particular goal.

My point is that this was expanded over time.

In effect, the goal became "let's reinvent IP was lots of ambitious
features" and that's a story template that never ends well. Or, in this
case, possible at all.

(I haven't re-read it from when it was developed by my recollection is
that even it was more ambitious than was needed, but that's just my
rogue opinion.)

While I realize that fragmentation was your starting point, my comments
were not meant to be about that one way or the other.

(However I'd had the impression that even IPv4 fragmentation is
problematic these days, so am surprised it's an issue for v6; but really
my comments were meant as broadly as the language indicates and not at
all about a particular like this.)

Post by Paul Vixie
we were volunteers, who worked on what we found deserving of our time.

(We can deal with the deceptive IETF myth of 'volunteer' some other time.)

The essence is that, yes, a 'strongly dominant' set of those working on
IPv6 chose to expand the scope beyond the original intent of the effort,
namely to simply expand the address space. My claim then and now is
that that greatly facilitated the problematic history that ensued.

Post by Paul Vixie
if the fragmentation differences between v4 and v6 were as you say a
form of scope creep,

but I didn't say that and I apologize for, apparently, not being clear
that my comments were not at all meant about fragmentation per se.
Simplistically, since v4 has fragmentation and had a really good reason
for needing it, I'd expect v6 to have it, absent a really good reason
not to.

Post by Paul Vixie
i was pushing for a simple expansion of the IP header so that we could
use source routing on all flows, to connect network 10 at each end,
through a series of tubes, really, that had unique IP addresses, so that

That's an amusing specific, for me. There was a meeting with Vint and
others discussing the challenge of testing IPv6, given that there was no
test lab large enough to make for an interesting case, and I suggested
tunnelling IP(v6)/IP(v4) to connect an aggregation of test labs. Vint
did not initially like the idea though I gather this changed over time.

Post by Paul Vixie
the path would become the identity. the dns portion of this design
looked a lot like what was later called 8+8. i was shouted down, as was
mike o'dell, so i harken to your suspicion that anything simple would've
been rejected.

cf my earlier posting that proposed taking alternative numbering scheme
out of the critical path for adoption.

At the time the topic of numbering had quite a lot of theory and
research proposals -- often with a great deal of personal fervor, but
without much detail to back them up or appreciation that they were
theory, not practice.


d/

[Paul Vixie]

Post by Paul Vixie

The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.

that word, "mandate," i don't think it means what you think it means.

cf, RFC 1726. That was not produced quickly nor in isolation nor by only
a tiny collection of wayward folk. It creates a mandate to work on a
particular problem to a particular goal.

ok, thanks for explaining. this is like the mandate every winner of
every political race claims, for the policies that got them elected.
like "the reagan mandate" of 1980, an election in which fewer than half
of eligible voters actually voted. that is to say, a fake news mandate.

had the ietf actually adhered to the limits of RFC 1726, we would not
have a radically different fragmentation model in ipv6 compared to ipv4.
so i think we can tell that not only the actual "internet engineers" of
the world, but also their chosen vehicle, were in no way constrained by
the thing you are calling a "mandate".

My point is that this was expanded over time.

my point is that such expansion was inevitable and should have been
expected and the people who ratified the mandate ought to have known better.

[Dave Crocker]

Post by Paul Vixie

Post by Paul Vixie

The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.

that word, "mandate," i don't think it means what you think it means.

cf, RFC 1726. That was not produced quickly nor in isolation nor by only
a tiny collection of wayward folk. It creates a mandate to work on a
particular problem to a particular goal.

ok, thanks for explaining. this is like the mandate every winner of
every political race claims,

It is nothing like that at all. It is like the word "requirement" in a
formal document meaning 'to require' and delegating the task of
satisfying the requirement(s) to some folk with what is sometimes called
a mandate.

Post by Paul Vixie
had the ietf actually adhered to the limits of RFC 1726, we would not
have a radically different fragmentation model in ipv6 compared to ipv4.

On this point of actual technical substance, we agree.

Post by Paul Vixie
so i think we can tell that not only the actual "internet engineers" of
the world, but also their chosen vehicle, were in no way constrained by
the thing you are calling a "mandate".

Field teams often go astray of their mandate. That doesn't make the
mandate not a mandate.

Post by Paul Vixie

My point is that this was expanded over time.

my point is that such expansion was inevitable and should have been
expected and the people who ratified the mandate ought to have known better.

"Inevitable" is such a dangerous word. In this case, you've made a leap
to its use that skips over so many complex human and social issues, it
looks more like a statement of religion than engineering.

d/

[Paul Vixie]

Post by Paul Vixie
so i think we can tell that not only the actual "internet engineers" of
the world, but also their chosen vehicle, were in no way constrained by
the thing you are calling a "mandate".

Field teams often go astray of their mandate. That doesn't make the
mandate not a mandate.

it was never a mandate on the people who weren't part of the discussions
of RFC 1726 or whose interests weren't aligned. so the answer to "when
is something not a mandate?" would include this.

Post by Paul Vixie

Post by Dave Crocker
My point is that this was expanded over time.

my point is that such expansion was inevitable and should have been
expected and the people who ratified the mandate ought to have known better.

"Inevitable" is such a dangerous word. In this case, you've made a leap
to its use that skips over so many complex human and social issues, it
looks more like a statement of religion than engineering.

i appreciate your candor, sir. a single counterexample from history
could mollify me. my own survey showed that whenever an equivalent
opportunity, motive and means were present, the result might as well
have been preordained.

[Dave Crocker]

Post by Paul Vixie

Post by Paul Vixie
so i think we can tell that not only the actual "internet engineers" of
the world, but also their chosen vehicle, were in no way constrained by
the thing you are calling a "mandate".

Field teams often go astray of their mandate. That doesn't make the
mandate not a mandate.

it was never a mandate on the people who weren't part of the discussions
of RFC 1726 or whose interests weren't aligned. so the answer to "when
is something not a mandate?" would include this.

Well, /that/ observation is a touchstone to rather more-interesting
questions about standards efforts, credibility and efficacy.

Post by Paul Vixie

Post by Paul Vixie

Post by Dave Crocker
My point is that this was expanded over time.

my point is that such expansion was inevitable and should have been
expected and the people who ratified the mandate ought to have known better.

"Inevitable" is such a dangerous word. In this case, you've made a leap
to its use that skips over so many complex human and social issues, it
looks more like a statement of religion than engineering.

i appreciate your candor, sir. a single counterexample from history
could mollify me. my own survey showed that whenever an equivalent
opportunity, motive and means were present, the result might as well
have been preordained.

We have lots of examples of efforts that retained control over scope,
did work in a timely fashion, and produced work that was useful in the
way originally intended. Consider the successful upgrade efforts, such
as the sequence of changes to IPv4 address schemes, BGP, SNMP, MIME,
SMTP extensions...

We do, of course, also have lots of examples that fail on one or more of
these points.

There do seem to be some factors that contribute to failure or
contribute to success, but I don't recall seeing them documented and
justified. Thaler's RFC 5218 is relevant here, but more for background
than actual prescription.

My model has been that a combination of participation by those who must
actually deliver the work to the marketplace -- so, development and ops
and even business/marketing -- with a very strong dose of urgency, tend
to produce more pragmatic and timely results.


d/

[Craig Partridge]

Post by Dave Crocker

Post by Paul Vixie
ony li said that ipv6 was too little, too soon. this was a play on
words, because the usual complaint is "too little, too late". tony was
right, even moreso than i realized at the time. we specified a lot of
things that didn't work and had to be revised or thrown out -- because
we did not know what we needed and we thought we were in a hurry.

...
This was mere scope creep. Whether because of second system syndrome or
a failure to sufficiently feel the urgency of getting something fielded
and working sooner rather than later, I don't know. But I suspect it
was both.

As one of the authors of the IPng Requirements (RFC 1726), I'm going to risk
piping up here. The requirements in RFC 1726 were for an IPv4 replacement
that scaled -- if you read RFC 1726 (and I just went back and did) it is
carefully
only requiring what we'd already achieved in IPv4 -- with the exception of
saying
we'd like something that configured better than DHCP.

So Tony's closer to right. We were so scared when we wrote that document
that
we only had a few years that we wrote remarkably conservative requirements.
(I'd actually pushed to be more ambitious and got shot down).

Craig

[Scott O. Bradner]

…

Post by Dave Crocker
The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.

the mission, as given, was stated in RFC 1719 section 3

Post by Dave Crocker
This was mere scope creep. Whether because of second system syndrome or
a failure to sufficiently feel the urgency of getting something fielded
and working sooner rather than later, I don't know. But I suspect it
was both.

fwiw - it was a community decision on mission (creep) - see RFC 1726 (the
criteria document called for in RFC 1719)

Scott

[Noel Chiappa]

From: Dave Crocker
The original mandate was for more address space. All the other
'features' that were attempted went beyond that mandate.
This was mere scope creep. Whether because of second system syndrome or
a failure to sufficiently feel the urgency of getting something fielded
and working sooner rather than later, I don't know.

How ironic. I felt at the time (and said so, vociferously, to the entire
community) that IPv6 was way too small/simple a change.

My reasoning was two-fold:

- If you wanted to get it deployed, it had to have major new capabilities as
incentives. With simply "more addresses", it was totally clear that IPv4+NAT
gave most of that, at an incredibly lower cost.

- If you're going to do an upgrade to a massive, world-wide system, that
upgrade ought to do as much as possible, because one doesn't often get a
chance to do something like that.

Oh well.

Noel

[Toerless Eckert]

IMHO, the fundamental issue is that we lost the speed of experimentation
and agile improvement of network infrastructure before 1994, with
the introduction of specialized,accelerated hardware, vendor software
development processes and "professional" network operations.

I don't think we'll ever recover even though its technically not too
difficult to see whats needed: actual virtual networks that can be evolved
and innovated with. Down to the NPU and packets schedulers. If every
network device was a VM-hosting compute system you could even think of getting
there.

Maybe we're going to get some 5% of whats needed via network slicing.
Judgemenet on the percentages is out. But its not going to be high.

Applications kinda went der othre way, having the ability for a lot more
agile innovation and experimentation through VMs in the last 10 years.
To some extend, that is also coming to an end with containeriziation
because it removes the lower layers of innovation and increases the lock in of
innovative applications into a maze of linux and ecosystem vendor peculiarities.
Not to speak of serverless compute. But at least the gilded society itself
will still be able to innovate faster than network equipment vendors.

Cheers
Toerless

Post by Paul Vixie
...

Now that IPv4 is truly hitting its limits, the main operational complaint
against IPv6 is that it's too different from IPv4. But the incentives are
finally shifting in IPv6's favour, like it or not.

i don't even know what i don't like anymore. but for the history books
tony li said that ipv6 was too little, too soon. this was a play on
words, because the usual complaint is "too little, too late". tony was
right, even moreso than i realized at the time. we specified a lot of
things that didn't work and had to be revised or thrown out -- because
we did not know what we needed and we thought we were in a hurry. we had
time, as history will show, to spend another ten years thinking about ipng.
where we are today is that fragmentation is completely hosed. pmtud does
not work in practice, and also cannot work in theory due to scale
(forget speed-- it's scale that kills.) the only reliable way to
communicate with ipv6 is to use a low enough MTU that it never exceeds
any link MTU. in practice that means an ipv6 payload, plus its headers,
has to fit in an ethernet packet, so, 1500 octets. you can special case
the on-link LAN scenario so that if you have 9000 octets available you
can use them -- but that's the only time you can use more than about
1200 octets for your payload.
this means one of ipv6's major claimed-up-front advantages, which is
that only endpoints will fragment rather than gateways doing so as in
ipv4, never came about. in fact, ipv6 is far worse than ipv4 in this
way, as we learned by using ip fragmentation on UDP/53 (my idea: bad!)
this also means that we're chained to the MTU of the second-generation
10-megabit ethernet, which was carefully sized to fit a bunch of radio
spectrum and cable length parameters which have never applied since
then. but the IEEE 802 people know they're stuck with 1500 forever,
since no next generation of ethernet can succeed without being able to
transparently bridge onto the previous generation.
history is hard, let's do math.

; (155*10^6) / (53*8)
~365566.03773584905660377358
; (10*10^6) / (1500*8)
~833.33333333333333333333
; (100*10^6) / (1500*8)
~8333.33333333333333333333
; (1000*10^6) / (1500*8)
~83333.33333333333333333333
; (10000*10^6) / (1500*8)
~833333.33333333333333333333
; (40000*10^6) / (1500*8)
~3333333.33333333333333333333
; (100000*10^6) / (1500*8)
~8333333.33333333333333333333

right, so ATM failed in the market for a lot of reasons (state is what
kills, not speed, like i said) but one of those reasons was that an OC3C
at line rate was carrying too many cells per second to be able to handle
all of their headers in then-current or even projected-soon electronics.
we were wrong, and ATM has been used at OC12C, OC48C, and i've even seen
OC192C and OC768C, truly a testament to human cussedness fit for a
bumper sticker or perhaps a t-shirt.
looks to me like less than half a 10GBE is just as bad, and that at
40GBE and 100GBE it's well beyond absurdity. thankful as we are for
moore's law, i regret like anything the inability to send large enough
packets in the WAN so that we don't all need a 100 kilowatt routers to
handle the headers.
ipv6's mindless and unnecessary early adoption of an unworkable
fragmentation regime has chained my applications and those of my
children and their children to the maximum size of a packet in a closed
10-megahertz radio network. yay us.
--
P Vixie
_______
internet-history mailing list
http://mailman.postel.org/mailman/listinfo/internet-history