Alternatives to Excel for tabular data? Minitab? Tableau?

Howard · May 30, 2018

truetype said:
You can learn all the R you need in two hours. After one day working with R, you'll wonder why you didn't try it years ago. And it's free.

That's very intriguing. Any pointers on how to proceed with doing so? Many thanks in advance.

tommcginnis · May 30, 2018

ryker said:
Do you know if RTD is supported by OpenOffice. At the moment I use Excel with Python (xlwings) for my trading and rely on RTD to get realtime prices (only used for PnL).

I looked again (just now) and didn't see anything. RTD, of course, is strickly MSExcel. There are other facilities, but you'd have to see how well they fit -- and the documentation is lacking, so you have to be willing to experiment. (FWIW, I have discovered things not-yet-documented, so there's a bit of fun in there..... if that's your idea of fun....) Sorry I can be more definitive, but it's been a while since I looked.

Let us know if you find out anything fun.

Howard · May 30, 2018

tommcginnis said:
All these replies echo what you've already received.
The thing is, we don't really know what you're doing.

Nor do we know how your Excel is prioritized by your computer -- or *what* computer you're running.

Yes, I understand that., It's difficult to get accurate advice when one can't provide accurate information about specifics (which I simply can't...).

I have a Dell 7000 laptop with 16 GB DDR memory and 128 GB SSD hard-drive I bought just months ago. It could very well be that I have considerable potential for improvement here, but I think it's mostly the framework provided to me that's too slow.

traider · May 30, 2018

Howard said:
Actually, my data IS stored in SQL and I utilize an SQL database already. But I don't really have a clue as to exactly how it works. But basically, my calculations are done in C# and stored in a DB.

Excel is basically just used as a front-end to display the data and filter/sort it.

The calculations outside Excel are fairly fast. It's loading, refreshing and navigating in Excel which is slow. I have a hunch that it could have been set up better by those who programmed it.

Regardless, I still have the same predicament which is that I'm basically dependent on outside help whenever I get a new idea that I want implemented within this framework. I would love to be completely independent.

I'm an engineer by trade and no fool, so I can look at code and have a basic understanding of some of it, but I currently feel it's a long stretch from there and to proficiently work in code myself.

If you know C#, python is fast to pick up. I think courses on coursera udemy youtube, etc should be decent

But even faster for you is to just use an excel to call into C# code. Used to do some time ago, i cant remember the exact details but it is fairly simple to link them up.
You don't even have to bother with python if you want to be productive fast and focus on strategy testing

truetype · May 30, 2018

Howard said:
That's very intriguing. Any pointers on how to proceed

Install R and (and perhaps also RStudio) and work through the freely-available Venables & Smith intro.

userque · May 30, 2018

Howard said:
Very interesting. I will do some Googling on these and see what I can discover on my own. If you're inclined to do so, please share some of your own experience (if you have used them yourself).

How is the data stored? Can I load data from my Excel sheets?

Yes, I've used both, but I needed to code a custom machine learning algo...and that was faster to do in a computer language.

You can store/load data anyway you like, excel, csv, binary, database, txt, etc.
---------
For example, to work with a spreadsheet file (excel, csv, etc.), you'd select a 'loader' module and put it on the workspace. You'd then open it up in order to set the parameters. One parameter could be the location of said file. Selecting that would cause a standard windows file explorer window to open. Simply select the file.

What do you want to do with the file? Let's say you want to sort the data in it.

So you'd grab a 'sort' module and put it in the workspace.

Now, connect the loader to the sorter. (The data 'flows' through these connections, making it easier to visualize workflow). Then set the sorter parameters, telling it which column to sort and how to sort it.

Then, connect the output from the sorter to 'the display.' Then run it.

This is a simple example based from my memory only, mind you. But there are plenty of videos and tutorials.

Here's one:

Howard · May 30, 2018

traider said:
If you know C#, python is fast to pick up. I think courses on coursera udemy youtube, etc should be decent

But even faster for you is to just use an excel to call into C# code. Used to do some time ago, i cant remember the exact details but it is fairly simple to link them up.
You don't even have to bother with python if you want to be productive fast and focus on strategy testing

I wish...

My current set-up was programmed for me by a company.

truetype said:
Install R and (and perhaps also RStudio) and work through the freely-available Venables & Smith intro.

Thanks, man.

Those dense 100 pages look a bit more extensive than 2 hours and a day of work though.

But I will take a look later. When I have time (which I don't seem to have these days...).

Howard · May 30, 2018

userque said:
Yes, I've used both, but I needed to code a custom machine learning algo...and that was faster to do in a computer language.

You can store/load data anyway you like, excel, csv, binary, database, txt, etc.
---------
For example, to work with a spreadsheet file (excel, csv, etc.), you'd select a 'loader' module and put it on the workspace. You'd then open it up in order to set the parameters. One parameter could be the location of said file. Selecting that would cause a standard windows file explorer window to open. Simply select the file.

What do you want to do with the file? Let's say you want to sort the data in it.

Wow. This looks really impressive. And could be very useful to me.

After loading the data - do you know if it's stored in the program?

Obviously, the answer depends on a person's background, but how complicated is this program to learn and get up to speed with?

Is it easy to create charts and stuff with this?

In addition to sorting my data set EOD, I'd also be interested in creating a dashboard for live data (intraday) for comparative analysis, etc.

R1234 · May 30, 2018

tommcginnis said:
All these replies echo what you've already received.
The thing is, we don't really know what you're doing.

Nor do we know how your Excel is prioritized by your computer -- or *what* computer you're running. If you're flogging some old horse of a computer -- or have it set up with a big block of CPU and RAM reserved for some other software -- no change in stat platform will improve on that. (For example, installing a $60 SSD might make your spreadsheet blazingly fast, if it writes back-and-forth to your hard drive.)

That said, I am in the same boat as you, however. I scrape 71 SPX and ES strikes over 6 expiries into an OpenOffice spreadsheet, and do a bunch of analysis from there. The spreadsheets have topped 2.5MB, and lately have gotten slower in loading and recalcs. ((Since a lot of the spreadsheet is for general research, I have thought about trimming it into two spreadsheets -- a full-bore version, and a chopped-down "sport" version for trading.))

That said, I went with Open/LibreOffice, because
1) I hate Microbloat.
2) OO/LO is more stable in operations.
3) OO/LO carries a much smaller footprint, re CPU, RAM, and storage.
4) OO/LO is VERY flexible with regard to backwards compatibility.
5) OO/LO is flexible with regard to other spreadsheet formats.
(That said, there is not 100% correspondence between OO/LO functions and Excel functions, and your spreadsheet make (likely) require some work to operate 100% in OO/LO.)

Last thought: if you were to switch, go with LibreOffice at this point, as the best supported/updated.

Major limitation of LibreOffice is they limit you to 1000 columns. And Libre uses that awful macro language that is similar to the one in OpenOffice.

userque · May 30, 2018

Howard said:
Wow. This looks really impressive. And could be very useful to me.

After loading the data - do you know if it's stored in the program?

Obviously, the answer depends on a person's background, but how complicated is this program to learn and get up to speed with?

Is it easy to create charts and stuff with this?

In addition to sorting my data set EOD, I'd also be interested in creating a dashboard for live data (intraday) for comparative analysis, etc.

It can store all, or some of the data in memory...depending upon your memory capacity, data size, software settings, and software version.

It's not complicated. The tutorial is built in and walks you through everything. The less you know about programming, the easier I suspect it'll be.

It is easier to create charts/graphs with it than it is to do with Excel, imo.

See below:

https://community.rapidminer.com/t5/RapidMiner-Studio-Forum/real-time-data/td-p/6587
https://community.rapidminer.com/t5/RapidMiner-Studio-Forum/Creating-a-dashboard/td-p/31554