<div>Hello, Kana,</div><div>I might be wrong but as far as I remember, the filters related with RecursiveGaussian aren't compatible with the streaming filter :(</div><div><br></div><div>Regards</div><br><div class="gmail_quote">
On Thu, Mar 17, 2011 at 11:54 PM, Arunachalam Kana <span dir="ltr"><<a href="mailto:Kana.Arunachalam@fh-wels.at">Kana.Arunachalam@fh-wels.at</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="DE-AT" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-GB">Hi ITK Users,</span></p><p><span lang="EN-GB">I need some advice on memory management and speed. </span></p><p><span lang="EN-GB">My system details: 64 bit windowsXP system and 4Gb RAM.</span></p>
<p><span lang="EN-GB">My goal: To obtain eigen vectors for 100Mb unsigned short image. Eigen vectors not for the whole image but for certain regions (approx occupies 40% of image).</span></p><p><span lang="EN-GB"> </span></p>
<p><span lang="EN-GB">I have to do some preprocessing too.</span></p><p><span lang="EN-GB">The pipeline is :</span></p><p><span lang="EN-GB">Original image -> discretegaussian -> gradientmagnitude -> HessianRecursiveGaussianImageFilter->symmetric eigenvalue analysis -> finally 3 vector image</span></p>
<p><span lang="EN-GB">For the above pipeline for a 100Mb image, i am running out of RAM.</span></p><p><span lang="EN-GB">After reading itkmails, i decided to use streamfilter to stream part of the image so that the RAM usage is low. I only tested half way and the RAM was overloaded.</span></p>
<p><span lang="EN-GB"> </span></p><p><span lang="EN-GB">I am using streamimagefilter to reduce the memory usage. After calculation of gradientmagnitude i have used 1.08Gb RAM. After this i need 1.2Gb RAM</span></p><p><span lang="EN-GB">For hessianimageoutput, so by using streamfilter i thought i would be reaching 2.5Gm RAM but i reach approx 6Gb(virtual memory is used).</span></p>
<p><span lang="EN-GB"><span>1.<span style="font:7.0pt "Times New Roman""> </span></span></span><span lang="EN-GB">Am i doing something wrong with the stream filter ? (code given below)</span></p><p><span lang="EN-GB"><span>2.<span style="font:7.0pt "Times New Roman""> </span></span></span><span lang="EN-GB">I tried another option: As i do not need hessian for the whole image, I used discretehessianfunction for hessian calculation but it is 30 times slower that the recursivehessian. Is discretegaussianimagefunction multithreaded? </span></p>
<p><span lang="EN-GB"><span>3.<span style="font:7.0pt "Times New Roman""> </span></span></span><span lang="EN-GB">Is there any other way to achieve speed vs RAM compromised solution?</span></p><p><span lang="EN-GB"><span>4.<span style="font:7.0pt "Times New Roman""> </span></span></span><span lang="EN-GB">The 100Mb data set is test data. The real data is 10Gb for which i will use 64 bit linux system (opensuse 11.2) . will the pipeline be executed in linux too?</span></p>
<p><span lang="EN-GB">Below is my code and after application of filter show the RAM usage:</span></p><p><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> char *infilename = "StreamTest.mhd"; </span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::CovariantVector<float,3> VectorPixelType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::Vector< VectorPixelType, 3 > EV_PixelType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> //Image type</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB"> typedef itk::Image<unsigned short, 3> InputImageType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::Image<float, 3> FloatImageType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::Image<EV_PixelType,3> EVImageType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB" style="color:red">**** RAM usage = 674Mb****</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> //reader initialisation and reading file</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB"> typedef itk::ImageFileReader<InputImageType> ReaderType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> ReaderType::Pointer reader = ReaderType::New();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> reader->SetFileName( infilename );</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> reader->Update();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> InputImageType::Pointer inImage = reader->GetOutput();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB" style="color:red">**** RAM usage = 783Mb****</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::DiscreteGaussianImageFilter<InputImageType, FloatImageType> RGIFType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> RGIFType::Pointer gaussfilter = RGIFType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> gaussfilter->SetInput(inImage);</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> gaussfilter->SetVariance(4.0);</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> gaussfilter->SetMaximumError(0.01);</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::CastImageFilter<FloatImageType,InputImageType> CIFType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> CIFType::Pointer castfilter = CIFType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> castfilter->SetInput(gaussfilter->GetOutput());</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::GradientMagnitudeImageFilter<InputImageType,FloatImageType> GMIFType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> GMIFType::Pointer gmfilter = GMIFType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> gmfilter->SetInput(castfilter->GetOutput());</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> gmfilter->Update();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB"> FloatImageType::Pointer gmImage = gmfilter->GetOutput();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB" style="color:red">**** RAM usage = 1290Mb****</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> gaussfilter->UnRegister();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
castfilter->UnRegister();</p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span style="color:red"> **** RAM usage = 1080Mb****</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
</p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"> <span lang="EN-GB">typedef itk::HessianRecursiveGaussianImageFilter<FloatImageType> HessianRecursiveGaussianFilterType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB"> typedef HessianRecursiveGaussianFilterType::OutputImageType HessianImageType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> HessianRecursiveGaussianFilterType::Pointer hessianfilter = HessianRecursiveGaussianFilterType::New();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> hessianfilter->SetInput(gmImage);</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> hessianfilter->SetSigma(1);</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> typedef itk::StreamingImageFilter<HessianImageType,HessianImageType> StreamerType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> StreamerType::Pointer streamer = StreamerType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> streamer->SetInput( hessianfilter->GetOutput() );</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> streamer->SetNumberOfStreamDivisions( 20 );</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> streamer->Update();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB" style="color:red">**** RAM usage = 3600Mb**** after few seconds it came to **** RAM usage = 6120Mb****</span></p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB"> </span></p><p><span lang="EN-GB">Thank you in advance.</span></p><p><span lang="EN-GB">Regards,</span></p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Kana Arunachalam Kannappan</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">
Research Associate</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">FH OÖ Forschungs & Entwicklungs GmbH</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Stelzhamer Strasse 23,</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">
4600 Wels,</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Austria.</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Phone: +43 (0)7242 72811 -4420</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">
<a href="mailto:kana.arunachalam@fh-wels.at" target="_blank">kana.arunachalam@fh-wels.at</a></p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt"><a href="http://www.fh-ooe.at" target="_blank">www.fh-ooe.at</a>; <a href="http://www.3dct.at" target="_blank">www.3dct.at</a></p>
<p class="MsoNormal"> </p></div></div><br>_____________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at<br>
<a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Kitware offers ITK Training Courses, for more information visit:<br>
<a href="http://www.kitware.com/products/protraining.html" target="_blank">http://www.kitware.com/products/protraining.html</a><br>
<br>
Please keep messages on-topic and check the ITK FAQ at:<br>
<a href="http://www.itk.org/Wiki/ITK_FAQ" target="_blank">http://www.itk.org/Wiki/ITK_FAQ</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.itk.org/mailman/listinfo/insight-users" target="_blank">http://www.itk.org/mailman/listinfo/insight-users</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Sergio Vera<br><br> Alma IT Systems<br> C/ Vilana, 4B, 4º 1ª<br> 08022 Barcelona<br> T. (+34) 932 380 592<br> <a href="http://www.alma3d.com">www.alma3d.com</a><br>