Ttdfjt

I’m in second year of my PhD in a environmental-science laboratory and there aren’t many experts on machine learning here.

In one part of my research I should use a multilayer perceptron to build a model from a series of inputs that I determined before. So for this part I suggested to use a deep learning framework, specifically Keras with Tensorflow or CNTK, but in the lab my supervisors told me that for a real application they don’t want to depend of a framework from Google or Microsoft and they prefer to develop they own tools that won’t change in the time if there’s for example an upgrade or something like that. One researcher created a neutral net and he obtained good results, but when we are interested in make some fine tuning, his code is more complex to update or to test new things like an dropout for example.

I started to work with those frameworks having in mind the reproducibility because I only have to use the functions from Keras and define the parameters. I think other researchers who want to test/use my model should use the configuration I suggest and don’t try to understand the code I provide. So I obtained some good results and am now in the phase of fine tuning, but the work of the other researcher has a slightly better results but after a regularisation step that I didn’t try yet.

So this researcher told me that I should abandon my work with Keras and start using his code. He also told me that Keras is just some fancy tool and in research is not required to have a very precise model, just good results and use the code that works.

So my question is this: Is it right what this researcher told me? Is Keras actually used in research? If there are some frameworks that are developed, why not use them instead of rediscovering the wheel, am I right?

In general any tool that many researchers use is by definition useful/helpful.

For the particular case of Keras and other neural network frameworks (like PyTorch, TensorFlow, etc), a lot of people use them. You can see this by reading papers in the topic, as it is usually mentioned which framework the implementation is made in. You can also check "popularity" by counting references, for example for keras and tensorflow, both have thousands of citations.

In general you should prefer well known tools that are used in the field. These are validated and you can at least have a degree of trust that the results they produce are correct (minus user mistakes, of course). Implementing a neural network framework from scratch is not easy and any reviewer can point out that there could be programming errors and/or mistakes. Also as you mention there is the advantage of reproducibility, as it is much more likely that people can run your code.

Also note that many frameworks were actually made by researchers, so they are specifically targeted for other researchers, and their design is made for easy experimentation. In keras for example, it is easy to implement some custom functionality as a loss or layer, so you can integrate it with another network design. A framework can have another target userbase such as deployment for low power platforms or computers, which Keras doesn't.

Finally, you have to consider that since a lot of people use Keras, there is a large community around it from where you can get support, such as Stack Overflow or the keras-users google group. Using custom code will make this more difficult or impossible.

If you want not to depend on a specific framework, you can always export networks in ONNX format so you can use it with another framework or your own code.

I've spent a decent amount of time trying to debug the differences between the same tests run on Keras and on frameworks custom built to run on hardware. This comes up when the hardware results are garbage, or underperforming. The Keras results are reliable enough to use as a benchmark for these purposes, and improving on them isn't considered.

But I've also run into version compatibility issues with Keras, so it's not an unfounded concern. There have been times when I've tried to run older networks with deprecated layers, which can be addressed by using an older version of Keras which is fairly simple. Or if old & new layers are needed together, it's possible to re-add the older layers yourself, as well as custom layers in general. Note that deprecation doesn't happen overnight, and there seems to be a very long warning time before for updates that may break something. So it's a concern, but not really a big problem.