An Inconsistency in Numerical Approximation











up vote
6
down vote

favorite












Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?










share|cite|improve this question




















  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    2 days ago






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    2 days ago










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    2 days ago










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    2 days ago






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    2 days ago

















up vote
6
down vote

favorite












Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?










share|cite|improve this question




















  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    2 days ago






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    2 days ago










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    2 days ago










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    2 days ago






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    2 days ago















up vote
6
down vote

favorite









up vote
6
down vote

favorite











Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?










share|cite|improve this question















Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?







arithmetic approximation fractions






share|cite|improve this question















share|cite|improve this question













share|cite|improve this question




share|cite|improve this question








edited yesterday









Servaes

21.1k33792




21.1k33792










asked 2 days ago









SZN

2,713620




2,713620








  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    2 days ago






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    2 days ago










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    2 days ago










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    2 days ago






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    2 days ago
















  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    2 days ago






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    2 days ago










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    2 days ago










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    2 days ago






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    2 days ago










4




4




en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago




en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
2 days ago




5




5




This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago




This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
2 days ago












relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago




relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
2 days ago












@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago




@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
2 days ago




1




1




Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago






Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
2 days ago












3 Answers
3






active

oldest

votes

















up vote
6
down vote



accepted










It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$



Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






share|cite|improve this answer





















  • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
    – SZN
    2 days ago




















up vote
1
down vote













The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






share|cite|improve this answer




























    up vote
    1
    down vote













    There is no paradox.



    When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



    But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






    share|cite|improve this answer























    • I don't recall claiming there was a paradox, only a numerical issue.
      – SZN
      2 days ago










    • No worries friend :-)
      – SZN
      2 days ago











    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    6
    down vote



    accepted










    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






    share|cite|improve this answer





















    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      2 days ago

















    up vote
    6
    down vote



    accepted










    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






    share|cite|improve this answer





















    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      2 days ago















    up vote
    6
    down vote



    accepted







    up vote
    6
    down vote



    accepted






    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






    share|cite|improve this answer












    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.







    share|cite|improve this answer












    share|cite|improve this answer



    share|cite|improve this answer










    answered 2 days ago









    Robert Israel

    314k23206453




    314k23206453












    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      2 days ago




















    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      2 days ago


















    Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
    – SZN
    2 days ago






    Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
    – SZN
    2 days ago












    up vote
    1
    down vote













    The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






    share|cite|improve this answer

























      up vote
      1
      down vote













      The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






      share|cite|improve this answer























        up vote
        1
        down vote










        up vote
        1
        down vote









        The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






        share|cite|improve this answer












        The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered 2 days ago









        José Carlos Santos

        142k20112208




        142k20112208






















            up vote
            1
            down vote













            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






            share|cite|improve this answer























            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              2 days ago










            • No worries friend :-)
              – SZN
              2 days ago















            up vote
            1
            down vote













            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






            share|cite|improve this answer























            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              2 days ago










            • No worries friend :-)
              – SZN
              2 days ago













            up vote
            1
            down vote










            up vote
            1
            down vote









            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






            share|cite|improve this answer














            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.







            share|cite|improve this answer














            share|cite|improve this answer



            share|cite|improve this answer








            edited 2 days ago

























            answered 2 days ago









            Yves Daoust

            122k668217




            122k668217












            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              2 days ago










            • No worries friend :-)
              – SZN
              2 days ago


















            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              2 days ago










            • No worries friend :-)
              – SZN
              2 days ago
















            I don't recall claiming there was a paradox, only a numerical issue.
            – SZN
            2 days ago




            I don't recall claiming there was a paradox, only a numerical issue.
            – SZN
            2 days ago












            No worries friend :-)
            – SZN
            2 days ago




            No worries friend :-)
            – SZN
            2 days ago


















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Quarter-circle Tiles

            build a pushdown automaton that recognizes the reverse language of a given pushdown automaton?

            Mont Emei