Please navigate to the bottom of the page for Table of Contents

Monday, January 21, 2013

Distributed vs Parallel computing


In this post I will provide a very high level overview of Distributed versus Parallel computing.

Distributed computing refers to the study of distributed systems to solve complex or time consuming problems, broken down to small tasks, across multiple computers (nodes) each of which has its own memory and disk.
In addition, the distributed system has additional constraints such as fault tolerance (individual nodes may fail), unknown structure (the network topology, etc.  may not be known or well defined) and decoupled (individual nodes may not have knowledge of entire system). The key to distributed computing is that there are many small nodes processing and executing tasks without knowing the broader system.

Parallel computing, on the other hand, is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel").

The same system may be characterized both as "parallel" and "distributed"; the processors in a typical distributed system run concurrently in parallel. The key difference between the two is that in parallel computing, all processors may have access to a shared memory to exchange information between processors and in distributed computing, each processor has its own private memory and information is exchanged by passing messages between the processors.

22 comments:

  1. This is very incorrect of course, confusing parallel processing with presence or absence of shared memory. Most and best parallel computing use MPI which is exactly passing messages. While shared-memory techniques like OpenMP had limited success.

    ReplyDelete
  2. Very good informative article. Thanks for sharing such nice article, keep on up dating such good articles.

    Best Digital Transformation Services | DM Services | Austere Technologies

    ReplyDelete
  3. Hi Nikhil,

    In total awe…. So much respect and gratitude to you folks for pulling off such amazing blogs without missing any points on the Distributed vs Parallel computing. Kudos!


    I know that you may answer as already in such case that you noticed that I open case 4914182641 in support center and that I need to wait their answer. But when I opened it, I start to search such problems with overdue payments on forums. And I found that everytime people said that they not get answer from support (from 48 hours to 1 week). I'm very affraid that in my case support can not react same as I read it from forums. And that my account can be closed. It's really little sum (approx. 8 EUR for february) and very big time of use (many years). I create domain's there and buckets. And it's very affraid to lost account such way. Can you, please, help me?

    Amazon S3 provides access to reliable and inexpensive data storage infrastructure. It is designed to make web-scale computing easier by enabling you to store and retrieve any amount of data, at any time, from within Amazon EC2 or anywhere on the web.

    But nice Article Mate! Great Information! Keep up the good work!

    Kind Regards,
    Kevin

    ReplyDelete
  4. Hi Mate,

    11/10!! Your blog is such a complete read. I like your approach with Distributed vs Parallel computing . Clearly, you wrote it to make learning a cake walk for me.

    We're currently in an infinite loop between sales and support, neither of whom seem to be able to understand a basic issue.

    We want to purchase some sizeable reserved instances but are told that the only way to pay is all at once with a credit card. No split payments, no offer to pay by check, no offer to pay by ACH, no offer to pay by wire. AWS Training

    Can someone explain to me how AWS serves enterprises if they only accept consumer methods of payment?

    Super likes !!! for this amazing post. I thinks everyone should bookmark this.

    Kind Regards,

    Irene Hynes

    ReplyDelete
  5. Merhaba,


    Three cheers to you ! Hooray!!! I feel like I hit the jackpot on Distributed vs Parallel computing


    I logged into console.aws.amazon.com today and now every few seconds a popup says I need to reload (and re-login). This has happened before after a long time of inactivity, but now it's happening so often I can't look at anything without having to reload and login. AWS Tutorial





    THANK YOU!! This saved my butt today, I’m immensely grateful.


    Obrigado,
    Ajeeth

    ReplyDelete
  6. I am very impressed after reading such a amazing post about cloud computing. Thanks for sharing such post. Cloud Computing Solutions Companies.

    ReplyDelete
  7. I was extremely dazzled by this post, this site has consistently been wonderful information. you particularly for such a fascinating post. oracle fusion finance training india

    ReplyDelete
  8. Great job, You are providing the best knowledge. It is really helpful and factual information for us and everyone to increase knowledge. Continue sharing your data. Thank you. microsoft dynamics houston tx

    ReplyDelete
  9. It’s my fortune to go to at this blog and realize out my required stuff that is also in the quality.
    interaction design agency

    ReplyDelete
  10. I was searching for the finest article on DPU. Because one of my friends told me about fungible that they are providing the best dpu service at a cheap rate. So i think i should visit their website and learn more about DPU.

    ReplyDelete
  11. I’m trampled by your contents carry on the wonderful work.
    logo creation services

    ReplyDelete
  12. The United Kingdom, with its many waves of immigration, is extremely culturally diverse. The UK is not only one of the leading financial centers in the world, but many other strong industries, which together form the fifth largest economy in the world and the second largest economy in Europe after Germany. http://www.confiduss.com/en/info/blog/article/uk-company-application/

    ReplyDelete
  13. This is important information which is shared by you. This info is meaningful and important for everyone to increase our knowledge about it. Always keep sharing this kind of information. Thank you. Read more info about Cloud PBX Baton Rouge

    ReplyDelete
  14. I was thinking to starting a business. But i don't have any idea on this. Then i found drops of knowledge. Here i got all the professional mentors who helped me so much. If you are also a new bee then you can seek from them.

    ReplyDelete
  15. It is a proficient article that you have shared here.Devops Company I got some different kind of information from your article which I will be sharing with my friends who need this info. Thankful to you for sharing an article like this.

    ReplyDelete
  16. Valuable info, this information is excellent and essential for everyone. I am very very thankful to you for providing this kind of information. Massey Ferguson Tractor

    ReplyDelete
  17. oh good! you bring us good content I'm glad to read. keep sharing it. - eicher 485

    ReplyDelete
  18. Thanks for publishing such excellent information. You are doing such a great job. This information is very helpful for everyone. Keep it up. Thanks for sharing it once again. John Deere Tractor

    ReplyDelete
  19. qq vip 303 เว็บไซต์ชั้นแนวหน้าระดับต้นๆของประเทศ pg slot เว็บไซต์ระบบออโต้ ที่ใช้ระบบอิเล็กทรอนิกส์ที่สุดยอดที่สุด เว็บไซต์ชี้แนะ สุดเริ่ด แจกจริง แจกมากมาย ลุ้นแจ็คพอร์ต

    ReplyDelete
  20. รับ เครดิต ฟรี ล่าสุด วิธีการรับเครดิตฟรีใหม่ล่าสุดและใช้ประโยชน์จากโอกาสที่น่าตื่นเต้นนี้ให้เกิดประโยชน์สูงสุด  PG SLOT เรียนรู้เกี่ยวกับแนวโน้มล่าสุด ข้อเสนอ

    ReplyDelete
  21. ร่ำรวย สล็อต สล็อตแตกง่าย เกมพนันออนไลน์ยอดฮิตยอดฮิตมากที่สุดในขณะนี้เพราะเหตุว่าเป็นเกม pg slot ที่มีตัวอย่างการเล่นที่ง่าย ไม่ซับซ้อน แตกต่างจากพนันอื่นๆอย่างแน่นอนที่เว็บเรา

    ReplyDelete