ruby on rails - How can I use Nokogiri to write a HUGE XML file? -

i have rails application uses delayed_job in reporting feature run large reports. 1 of these generates massive xml file , can take literally days in bad, old way code written. thought that, having seen impressive benchmarks on internet, nokogiri afford nontrivial performance gains.

however, examples can find involve using nokogiri builder create xml object, using .to_xml write whole thing. there isn't enough memory in zip code handle file of size.

so can use nokogiri stream or write data out file?

nokogiri designed build in memory because build dom , converts xml on fly. it's easy use, there trade-offs, , doing in memory 1 of them.

you might want using erubis generate xml. rather gather data before processing , keeping logic in controller, we'd rails, save memory can put logic in template , have iterate on data, should resource demands.

if need xml in file might need using redirection:

erubis options templatefile.erb > xmlfile 

this simple example, shows define template generate xml:

<%  asdf = (1..5).to_a  %> <xml>   <element> <% asdf.each |i| %>     <subelement><%= %></subelement> <% end %>   </element> </xml> 

which, when call erubis test.erb outputs:

<xml>   <element>     <subelement>1</subelement>     <subelement>2</subelement>     <subelement>3</subelement>     <subelement>4</subelement>     <subelement>5</subelement>   </element> </xml> 

---

edit:

the string concatenation taking forever...

yes, can because of garbage collection. don't show code example of how you're building strings, ruby works better when use << append 1 string when using +.

it might work better not try keep in string, instead write disk, appending open file go.

again, without code examples i'm shooting in dark might doing or why things run slow.