It would appear that since you are quoted as being "contractor" in the agreement you refer to, that indeed you are not formally an "employee". Are you being paid on a 1099 ?
If so, you should formally add a section to the agreement that you incorporate "Prior Art" into your work activities. As such, the Company may not have legal rights to certain intellectual properties that you have independently established before your work for them begins.
I am assuming by the wording in your original post that you are not being offered a formal position as a full-fledged W-2 employee of the Company.
Please elaborate.
Before I began a stint at a hedge fund, I included a "prior art" addendum into my employment contract and it stood. I also listed in general terms what such prior art included. "Intraday Automated Trading Algorithms", for example, is a general term.
If you are 1099 - IMO, sounds like they want their cake and eat it too. Contractors are generally employed based upon their established skills.