There's a reason why voxels are generally kept pretty big. Storing volume information requires huge amounts of memory usually. Consider a simple case where you want a 500 voxel view distance and you have a maximum height of 200 voxels. To allow the player to look around properly, you'd need to have all voxels within a 500 radius circle around the player loaded in. This is essentially a 1000x1000x200 volume of voxels, or two hundred MILLION voxels. Storing each voxel as an object will use at least 8 bytes per voxel (even an empty Java object uses 8 bytes), so you're already talking about 1000*1000*200*8 bytes, which is equal to 1.5GBs of memory right there. Let's say you want smaller voxels, so you cut each voxel into 8 smaller voxels. Well, now you have 8 times as many voxels, so that's 12GBs of memory you have now. There are a lot of tricks for reducing the memory usage of voxel worlds, and this should be your first focus (after you got basic rendering working).
Yeah, I will have to crunch some numbers.
Tessellation isn't really something you can use for voxel worlds effectively to get smaller voxels. Due to how most voxel world rendering techniques work, both the smallest and biggest piece of geometry you can render is a single voxel's face. The nature of voxel worlds makes it very hard to combine multiple voxels into a single surface without handling a lot of special cases, so it's an optimization that is rarely done. Tessellation allows you to dynamically divide one triangle or quad into more triangles or quads. This isn't really what you want to do in your voxel world, because it would require you to merge voxels ahead of time, then dynamically subdivide them again to get the original voxels back, which like I said is really hard to do. This means that tessellation doesn't really fill much of a function in voxel worlds. If none of what I just said makes sense to you, you should consider reading up more on tessellation and what it's actually used for nowadays, what its advantages and disadvantages are, etc.
Sorry, noob mistake. Too excited on new features
If you want to target consoles, then Java is definitely the wrong platform for you. Although there are ways to run Java games on consoles, there's currently no known way of achieving hardware acceleration for the graphics (or OpenGL at all), so they're essentially limited to simple 2D graphics.
Consoles - Just saying. I saw a large "Java Powered" with the logo, acknowledgements and all on my new box of PS4 last year. So I thought if there is a JVM in PS4 , why not put it to good use? And you mean Minecraft for consoles was coded in C++?
You need to look elsewhere if you want to run your games on consoles, probably the Unity engine and Unreal engine. If you just want to run your game on phones and tablets (as well as PC), then Java is a great choice. LibGDX is the go-to library for that, but LWJGL is seeing initial support for Android right now. Exciting times!
Basically, Java was NOT my first choice. It is almost one year since I formally started on working on my game, yet I have almost nothing but a few hundred lines of code and 9.5 months of failed experience . Also, I used the EXACT two engines you mentioned.
Last year, I started to work on my game. I first used Unreal Engine as some of my friends told me that it is the best game engine there is. For two months I learnt Blueprint and a little of C++ in Unreal. After that I started to work on voxel engines. Later I found out that the 'Engine in an Engine' was too much of a power hog (I am developing in a laptop). Then I moved on to Unity as I saw that there were a lot of voxel engine tuts for it. 4.3 months were spent on learning C#. I realized that the code depended on the syntax of the library, not the actual language (Unity C# was very similar to Java) . Then for two months I made two attempts to make my game.
Attempt 1 : FAILED
Attempt 2: I borrowed a bit from a voxel framework, but then I made a very complex pipeline and it was very difficult to :-
and add new features
Some good fellow at stack exchange told me that OpenGL is the best tool for my job. I chose Java with lwjgl as at that time I was being taught Java at school and I was very good at it.
So now I do not use any game engine and high level library because of this. So lwjgl for me.
Yes, I am very arrogant but sometimes arrogance helps. With luck, I might be the Steve Jobs of Java Game Development.
EDIT : I KNEW IT! Thanks everyone for supporting vao/vbos