An Inconsistency in Numerical Approximation
up vote
6
down vote
favorite
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
|
show 3 more comments
up vote
6
down vote
favorite
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago
|
show 3 more comments
up vote
6
down vote
favorite
up vote
6
down vote
favorite
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
arithmetic approximation fractions
edited yesterday
Servaes
21.1k33792
21.1k33792
asked 2 days ago
SZN
2,713620
2,713620
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago
|
show 3 more comments
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago
4
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago
5
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago
1
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago
|
show 3 more comments
3 Answers
3
active
oldest
votes
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
add a comment |
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
add a comment |
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
add a comment |
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
add a comment |
up vote
6
down vote
accepted
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
answered 2 days ago
Robert Israel
314k23206453
314k23206453
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
add a comment |
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
2 days ago
add a comment |
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
add a comment |
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
add a comment |
up vote
1
down vote
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
answered 2 days ago
José Carlos Santos
142k20112208
142k20112208
add a comment |
add a comment |
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
add a comment |
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
add a comment |
up vote
1
down vote
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
edited 2 days ago
answered 2 days ago
Yves Daoust
122k668217
122k668217
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
add a comment |
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
No worries friend :-)
– SZN
2 days ago
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago