<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Why does cuda now work well with the ITK pipeline?<div><br></div><div><br><div><div>On Feb 13, 2009, at 3:43 PM, Daniel Blezek wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>My lab currently makes heavy use of acceleration on the Cell, and we are<br>looking to do more with CUDA (hopefully OpenCL as it matures). Our take is<br>that accelerated code works best next to, and not inside ITK. For instance,<br>run an ITK pipeline, grab the output into some CUDA code, process using<br>CUDA, then back into ITK. This paradigm works quite well, though you'll<br>have to manage the pipeline updates manually.<br><br>This way ITK is used for what it's best at, and CUDA/OpenCL is used for what<br>it's best at.<br><br>$0.02,<br>-dan<br><br><br>On 2/12/09 2:19 PM, "Hua-Mei Chen" <<a href="mailto:huameichen0523@gmail.com">huameichen0523@gmail.com</a>> wrote:<br><br><blockquote type="cite">I have seen several discussions on this issue before in this emailing list.<br></blockquote><blockquote type="cite"><br></blockquote>My impression is that it is not possible or very difficult to combine CUDA<br><blockquote type="cite"><br></blockquote>and ITK for now. All I can say is that CUDA indeed is able to give you a<br><blockquote type="cite"><br></blockquote>significant speedup, but it strongly depends on the quality of your CUDA<br><blockquote type="cite"><br></blockquote>codes.<br><br>Chen<br><br><br>----- Original Message -----<br>From: "Marcel Weiss"<br><blockquote type="cite"><<a href="mailto:mweiss@cbs.mpg.de">mweiss@cbs.mpg.de</a>><br></blockquote>To: "Hua-Mei Chen" <<a href="mailto:huameichen0523@gmail.com">huameichen0523@gmail.com</a>><br>Cc:<br><blockquote type="cite"><<a href="mailto:insight-users@itk.org">insight-users@itk.org</a>>; <<a href="mailto:insight-developers@itk.org">insight-developers@itk.org</a>><br></blockquote>Sent: Thursday, February<br><blockquote type="cite">12, 2009 11:06 AM<br></blockquote>Subject: Re: [Insight-developers] [Insight-users] Cuda<br><blockquote type="cite">ITK<br></blockquote><br><br>As we ordered a new number crunching machine including a NVIDIA Tesla TM<br><blockquote type="cite"><br></blockquote>C1060 and we are planning to use it for the fast processing of our<br><blockquote type="cite"><br></blockquote>high-resolition 3D images, I“d be interested in some more statements and<br><blockquote type="cite"><br></blockquote>experiences on CUDA (pure and/or with ITK).<br><br>Would you suggest planning and<br><blockquote type="cite">programming new code or adjusting existing<br></blockquote>code?<br>I“m sure there is a tradeoff<br><blockquote type="cite">between man power needed and speed-up gained<br></blockquote>...<br><br>Marcel<br><blockquote type="cite">Weiss<br></blockquote>Dipl.-Inf.(FH)<br>PhD student<br>Department of Neurophysics<br>Max Planck<br><blockquote type="cite">Institute for Human Cognitive and Brain Sciences<br></blockquote><br>----- Original Message<br><blockquote type="cite">-----<br></blockquote>From: "Hua-Mei Chen" <<a href="mailto:huameichen0523@gmail.com">huameichen0523@gmail.com</a>><br>To: "Dan Mueller"<br><blockquote type="cite"><<a href="mailto:dan.muel@gmail.com">dan.muel@gmail.com</a>>, "Gaėtan Lehmann"<br></blockquote><<a href="mailto:gaetan.lehmann@jouy.inra.fr">gaetan.lehmann@jouy.inra.fr</a>>, "Badri<br><blockquote type="cite">Roysam" <<a href="mailto:roysam@ecse.rpi.edu">roysam@ecse.rpi.edu</a>><br></blockquote>Cc: "ITK Users" <<a href="mailto:insight-users@itk.org">insight-users@itk.org</a>>, "ITK<br><blockquote type="cite">Developers"<br></blockquote><<a href="mailto:insight-developers@itk.org">insight-developers@itk.org</a>><br>Sent: Thursday, February 12, 2009<br><blockquote type="cite">5:21:11 PM GMT +01:00 Amsterdam / Berlin /<br></blockquote>Bern / Rome / Stockholm /<br><blockquote type="cite">Vienna<br></blockquote>Subject: Re: [Insight-developers] [Insight-users] Cuda ITK<br><br>Well, we<br><blockquote type="cite">wrote some PURE CUDA code for our mutual information based<br></blockquote>non-rigid<br><blockquote type="cite">registration. It did give us a significant (> 100 times for some<br></blockquote>routines)<br><blockquote type="cite">speedup. But I have no idea how to use it with ITK.<br></blockquote><br>Chen<br><br><br>----- Original<br><blockquote type="cite">Message -----<br></blockquote>From: "Badri Roysam" <<a href="mailto:roysam@ecse.rpi.edu">roysam@ecse.rpi.edu</a>><br>To: "Dan Mueller"<br><blockquote type="cite"><<a href="mailto:dan.muel@gmail.com">dan.muel@gmail.com</a>>; "Gaėtan Lehmann"<br></blockquote><<a href="mailto:gaetan.lehmann@jouy.inra.fr">gaetan.lehmann@jouy.inra.fr</a>><br>Cc: "ITK<br><blockquote type="cite">Users" <<a href="mailto:insight-users@itk.org">insight-users@itk.org</a>>; "ITK<br></blockquote><blockquote type="cite">Developers"<br></blockquote><<a href="mailto:insight-developers@itk.org">insight-developers@itk.org</a>><br>Sent: Thursday, February 12, 2009<br><blockquote type="cite">8:50 AM<br></blockquote>Subject: Re: [Insight-users] [Insight-developers] Cuda ITK<br><br><br>We<br><blockquote type="cite">invested heavily in GPUs and CUDA in my lab a couple of years ago.<br></blockquote><blockquote type="cite">Our<br></blockquote>practical<br>experience has been that these things don't work quite as we had<br><blockquote type="cite">imagined<br></blockquote>from the<br>advertising. In other words, when we compiled our vessel<br><blockquote type="cite">segmentation codes<br></blockquote>on CUDA,<br>we found no speedup (in some case, it was slower<br><blockquote type="cite">than optimized compilation<br></blockquote>on the CPU.<br>It took a lot of work to get speedup<br><blockquote type="cite">over serial code. You have to rethink<br></blockquote>your<br>code to suit a streaming<br><blockquote type="cite">architecture, and spend time working out the data<br></blockquote>flows carefully.<br>Overall, we<br><blockquote type="cite">learned that the effort involved in building<br></blockquote>"GPU versions" of existing<br><blockquote type="cite">routines is<br></blockquote>manpower intensive. I am interested to know what others'<br><blockquote type="cite">experience has<br></blockquote>been.<br><br><br>Badri Roysam<br>Professor, Department of Electrical,<br><blockquote type="cite">Computer and Systems Engineering<br></blockquote>Associate Director, NSF Center for Subsurface<br><blockquote type="cite">Sensing & Imaging Systems<br></blockquote>(CenSSIS ERC)<br>Co-Director, Rensselaer Center for<br><blockquote type="cite">Open Source Software<br></blockquote>Rensselaer Polytechnic Institute<br>110 8th Street, Troy,<br><blockquote type="cite">New York 12180-3590, USA.<br></blockquote>Office(JEC 7010): 518-276-8067, Assistant:<br><blockquote type="cite">518-276-8525, Lab(JEC 6308):<br></blockquote>518-276-8207, Fax: 518-276-8715<br>Email:<br><blockquote type="cite">roysam@ecse.rpi.edu, Web: <a href="http://www.ecse.rpi.edu/~roysam">http://www.ecse.rpi.edu/~roysam</a><br></blockquote><br><br><br>----- Original<br><blockquote type="cite">Message -----<br></blockquote>From: Dan Mueller [<a href="mailto:dan.muel@gmail.com">mailto:dan.muel@gmail.com</a>]<br>To: Gaėtan Lehmann<br><blockquote type="cite">[<a href="mailto:gaetan.lehmann@jouy.inra.fr">mailto:gaetan.lehmann@jouy.inra.fr</a>]<br></blockquote>Cc: ITK Users<br><blockquote type="cite">[<a href="mailto:insight-users@itk.org">mailto:insight-users@itk.org</a>], ITK<br></blockquote><blockquote type="cite">Developers<br></blockquote>[<a href="mailto:insight-developers@itk.org">mailto:insight-developers@itk.org</a>]<br>Subject: Re: [Insight-users]<br><blockquote type="cite">[Insight-developers] Cuda ITK<br></blockquote><br><br><blockquote type="cite">Hi Gaetan,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I too have been following the<br></blockquote><blockquote type="cite">closely development of the GPGPU field<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://en.wikipedia.org/wiki/Gpgpu">http://en.wikipedia.org/wiki/Gpgpu</a><br></blockquote><blockquote type="cite">and waiting for ITK to jump on board<br></blockquote><blockquote type="cite">(unfortunately I have had little<br></blockquote><blockquote type="cite">spare time to spearhead this myself).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I<br></blockquote><blockquote type="cite">remember speaking to Luis at a t-con last year about ITK+CUDA and he<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">mentioned one of outcomes of the project was discovering that CUDA is<br></blockquote><blockquote type="cite">not<br></blockquote><blockquote type="cite">(yet) very portable (i.e. code that works on one NVIDIA card, does<br></blockquote><blockquote type="cite">not<br></blockquote><blockquote type="cite">necessarily work on another).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">If ITK is going to be GPGPU-enabled, then<br></blockquote><blockquote type="cite">perhaps the OpenCL standard<br></blockquote><blockquote type="cite">will provide more portability:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://en.wikipedia.org/wiki/OpenCL">http://en.wikipedia.org/wiki/OpenCL</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Obviously OpenCL is still new (version<br></blockquote><blockquote type="cite">1.0 approved December 8, 2008),<br></blockquote><blockquote type="cite">but I imagine we'll start seeing supporting<br></blockquote><blockquote type="cite">hardware soon<br></blockquote><blockquote type="cite"> <a href="http://www.nvidia.com/object/io_1228825271885.html">http://www.nvidia.com/object/io_1228825271885.html</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://news.softpedia.com/news/AMD-Takes-Up-OpenCL-1-0-Puts-More-Effort-on-ATI">http://news.softpedia.com/news/AMD-Takes-Up-OpenCL-1-0-Puts-More-Effort-on-ATI</a><br></blockquote><blockquote type="cite">-Stream-SDK-99578.shtml<br></blockquote><blockquote type="cite">and maybe even Intel's Larrabee<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://www.hpcwire.com/blogs/OpenCL_On_the_Fast_Track_33608199.html">http://www.hpcwire.com/blogs/OpenCL_On_the_Fast_Track_33608199.html</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(I<br></blockquote><blockquote type="cite">don't think NVIDIA or ATI has to date officially announced a chip<br></blockquote><blockquote type="cite">with<br></blockquote><blockquote type="cite">OpenCL support...I'd be happy to be proved wrong...)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Because much of this<br></blockquote><blockquote type="cite">is still speculative, it doesn't really factor<br></blockquote><blockquote type="cite">into your discussion about<br></blockquote><blockquote type="cite">what hardware to buy today... Just a<br></blockquote><blockquote type="cite">warning: while CUDA is the pioneer, it<br></blockquote><blockquote type="cite">may not be be around to see the<br></blockquote><blockquote type="cite">real action.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers, Dan<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">2009/2/11<br></blockquote><blockquote type="cite">Gaėtan Lehmann <<a href="mailto:gaetan.lehmann@jouy.inra.fr">gaetan.lehmann@jouy.inra.fr</a>>:<br></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Hi,<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">We are<br></blockquote></blockquote><blockquote type="cite">currently thinking to buy a few powerful workstations to run ITK<br></blockquote><blockquote type="cite"><blockquote type="cite">in<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">my<br></blockquote></blockquote><blockquote type="cite">lab.<br></blockquote><blockquote type="cite"><blockquote type="cite">I heard of nvidia's tesla cards, and I see that some experiments<br></blockquote></blockquote><blockquote type="cite">has<br></blockquote><blockquote type="cite"><blockquote type="cite">been<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">made to use them in ITK.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><a href="http://www.na-mic.org/Wiki/images/f/f9/Itk-gpu-meeting-Fall2007.ppt">http://www.na-mic.org/Wiki/images/f/f9/Itk-gpu-meeting-Fall2007.ppt</a><br></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Is<br></blockquote></blockquote><blockquote type="cite">there any update since fall 2007?<br></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Regards,<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Gaėtan<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite"><blockquote type="cite">Gaėtan Lehmann<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Biologie du Développement et de la Reproduction<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite">INRA de Jouy-en-Josas (France)<br></blockquote><blockquote type="cite"><blockquote type="cite">tel: +33 1 34 65 29 66 fax: 01 34 65 29<br></blockquote></blockquote><blockquote type="cite">09<br></blockquote><blockquote type="cite"><blockquote type="cite"><a href="http://voxel.jouy.inra.fr">http://voxel.jouy.inra.fr</a> <a href="http://www.mandriva.org">http://www.mandriva.org</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><a href="http://www.itk.org">http://www.itk.org</a> <a href="http://www.clavier-dvorak.org">http://www.clavier-dvorak.org</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_____________________________________<br></blockquote><blockquote type="cite">Powered by <a href="http://www.kitware.com">www.kitware.com</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Visit<br></blockquote><blockquote type="cite">other Kitware open-source projects at<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Please keep messages<br></blockquote><blockquote type="cite">on-topic and check the ITK FAQ at:<br></blockquote><blockquote type="cite"><a href="http://www.itk.org/Wiki/ITK_FAQ">http://www.itk.org/Wiki/ITK_FAQ</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Follow this link to subscribe/unsubscribe:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://www.itk.org/mailman/listinfo/insight-users">http://www.itk.org/mailman/listinfo/insight-users</a><br></blockquote><blockquote type="cite"><br></blockquote>__________________________<br><blockquote type="cite">___________<br></blockquote>Powered by <a href="http://www.kitware.com">www.kitware.com</a><br><br>Visit other Kitware open-source<br><blockquote type="cite">projects at<br></blockquote><a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br><br>Please keep<br><blockquote type="cite">messages on-topic and check the ITK FAQ<br></blockquote><blockquote type="cite">at:<br></blockquote>http://www.itk.org/Wiki/ITK_FAQ<br><br>Follow this link to<br><blockquote type="cite">subscribe/unsubscribe:<br></blockquote>http://www.itk.org/mailman/listinfo/insight-users<br><br>____<br><blockquote type="cite">___________________________________________<br></blockquote>Powered by www.kitware.com<br><br>Visit<br><blockquote type="cite">other Kitware open-source projects at<br></blockquote><blockquote type="cite"><br></blockquote>http://www.kitware.com/opensource/opensource.html<br><br>Please keep messages<br><blockquote type="cite">on-topic and check the ITK FAQ at:<br></blockquote>http://www.itk.org/Wiki/ITK_FAQ<br><br>Follow<br><blockquote type="cite">this link to<br></blockquote><blockquote type="cite">subscribe/unsubscribe:<br></blockquote>http://www.itk.org/mailman/listinfo/insight-developers<br><blockquote type="cite"><br></blockquote><br>_____________________________________<br>Powered by www.kitware.com<br><br>Visit<br><blockquote type="cite">other Kitware open-source projects<br></blockquote><blockquote type="cite">at<br></blockquote>http://www.kitware.com/opensource/opensource.html<br><br>Please keep messages<br><blockquote type="cite">on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ<br></blockquote><br>Follow<br><blockquote type="cite">this link to<br></blockquote><blockquote type="cite">subscribe/unsubscribe:<br></blockquote>http://www.itk.org/mailman/listinfo/insight-users<br><br><br>_______________________________________________<br>Powered by www.kitware.com<br><br>Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html<br><br>Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ<br><br>Follow this link to subscribe/unsubscribe:<br>http://www.itk.org/mailman/listinfo/insight-developers<br></div></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><div style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica">========================================================</font></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Bradley Lowekamp<span class="Apple-converted-space"> </span><span class="Apple-converted-space"> </span></font></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font class="Apple-style-span" face="Arial"><span class="Apple-style-span" style="font-family: Arial; "><span class="Apple-style-span" style="font-family: Arial; ">Lockheed Martin </span></span></font><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Contractor for</font></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Office of High Performance Computing and Communications</font></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica">National Library of Medicine<span class="Apple-converted-space"> </span></font></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica"><a href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a></font></p><br class="Apple-interchange-newline"></span></div></span> </div><br></div></body></html>