Improve performance for Frame export to CSV

Description

600Mx4 frame creation is fast

{{createFrame {"dest":"export_target","rows":"600000000","cols":"3","seed":7595850248774472000,"seed_for_column_types":-1,"randomize":true,"value":0,"real_range":100,"categorical_fraction":0.1,"factors":5,"integer_fraction":0.5,"binary_fraction":0.1,"binary_ones_fraction":0.02,"time_fraction":0,"string_fraction":0,"integer_range":1,"missing_fraction":0.01,"response_factors":2,"has_response":true}
}}

but CSV creation is super slow

{{exportFrame "export_target", "/tmp/export_target.csv", overwrite: true
}}takes 20+ minutes (edited)

Assignee

Michal Kurka

Fix versions

Reporter

Michal Kurka

Support ticket URL

None

Labels

None

Affected Spark version

None

Customer Request Type

None

Task progress

None

ReleaseNotesHidden

None

CustomerVisible

No

Priority

Major
Configure