Crosspost r/datascience

Suppose you’re working on a machine learning/coding contest and through your own research come up with a technique that is say, 4% better than the best thing anyone has tried (just pulling numbers out of the air here).

At what point is it better to not claim the prize on said contest and just keep the method secret? At what point should you publish it? Does it ever make sense to just use it in your own capacity analyzing data for companies as a 3rd party?

I mean I’m sure it’s all dependent on the money involved but one has to wonder where the breaking point is. What can you make as an independent 3rd party willing to do analysis with proprietary software you aren’t releasing?

In such a case, how could you ever provide confidence enough that the methods work? Also, how would you bill if essentially the time spent is mainly runtime? I’d love to hear any speculation, stories, industry standard behavior or history around this sort of thing.