Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (108)
games submitted by our members
Games in WIP (536)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Default XML parser grabs DTD from w3.org everytime  (Read 1449 times)
0 Members and 1 Guest are viewing this topic.
Offline CommanderKeith
« Posted 2012-01-05 20:25:04 »

Hey have you guys come across this problem? It's where the default java XML parser grabs the xml file's DTD from W3.org every single time it runs. I've spent the last 3 days trying to figure out why my app takes so long to load and it's because of this problem. Unbeknownst to me my app was actually getting the DTD from w3c's site each time, which caused a delay of about 30 seconds... Geez that's frustrating!

And smart people are having this problem too:
http://weblogs.java.net/blog/cayhorstmann/archive/2011/12/12/sordid-tale-xml-catalogs

So if i leave out this line from the XML file:

1  
<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">


then the javax.xml.parsers.SAXParser parses the file straight away.

But if I do that then the proper DTD is not used, so the proper solution is to setup the SAX parser like this (http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic/#comment-376):
1  
2  
SAXParserFactory factory = SAXParserFactory.newInstance();
factory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);

That line is not documented or mentioned anywhere on oracle.com except in the 376th comment on that w3.org blog post... gah!

Apparently W3 serve up 100 million dtd downloads/day, and the w3 guy says in the comments that 1/4 of these are from java apps:
http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic/#comment-359

I couldn't believe how silly this problem is so I felt the need to air my frustration  Tongue

Online Riven
Global Moderator

JGO Overlord


Medals: 751
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #1 - Posted 2012-01-05 20:34:01 »

It's indeed one of the biggest design flaws ever. You can bring down tens of thousands of applications by attacking this single point of failure.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline CommanderKeith
« Reply #2 - Posted 2012-01-05 20:41:02 »

It's bizarre, i would never have guessed that a simple XML parse would hook my app up to some random website.

How did you learn about the flaw?

It doesn't seem like a well-known problem. I googled 'SAX pause', 'xml delay', 'xml SAX stall', and many variations but couldn't find anything which indicated that this was my problem.

That w3 blog post was sometimes in the hits, but of course I never read so far down in the comments to see the solution.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Online Riven
Global Moderator

JGO Overlord


Medals: 751
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #3 - Posted 2012-01-05 20:46:39 »

How did you learn about the flaw?
Coincidence, I just stumbled upon a webpage about it a few years ago.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline pjt33
« Reply #4 - Posted 2012-01-05 23:16:25 »

I haven't come across this in Java, because I use an XML parser which doesn't check against the DTD, but I have come across it in .Net. I solved it there by downloading the DTDs and then hacking the XML files before putting them through the parser.
Offline aazimon
« Reply #5 - Posted 2012-01-06 00:34:09 »

Try using Dom4j instead.
Offline CommanderKeith
« Reply #6 - Posted 2012-01-07 02:33:00 »

I haven't come across this in Java, because I use an XML parser which doesn't check against the DTD, but I have come across it in .Net. I solved it there by downloading the DTDs and then hacking the XML files before putting them through the parser.
I tried using an xml file without the dtd doctype declaration but then the special entities like non breaking space &nbsp; would throw errors.

Offline CommanderKeith
« Reply #7 - Posted 2012-01-07 09:20:15 »

So i switched from using SAX to DOM and ran into similar troubles. I found this project which has worked well:

http://code.google.com/p/java-xhtml-cache-dtds-entityresolver/

Offline pjt33
« Reply #8 - Posted 2012-01-07 10:20:06 »

I haven't come across this in Java, because I use an XML parser which doesn't check against the DTD, but I have come across it in .Net. I solved it there by downloading the DTDs and then hacking the XML files before putting them through the parser.
I tried using an xml file without the dtd doctype declaration but then the special entities like non breaking space &nbsp; would throw errors.
Yes, that's why I mentioned downloading the DTDs. The hack was to remove the public DTD references and replace them with system ones.
Offline CommanderKeith
« Reply #9 - Posted 2012-01-07 17:08:46 »

Ah i see. It's so bizarre that this is not done by default in the java xml libraries, and that tutorials do not show how to do it.


Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

CogWheelz (18 views)
2014-07-30 21:08:39

Riven (23 views)
2014-07-29 18:09:19

Riven (15 views)
2014-07-29 18:08:52

Dwinin (12 views)
2014-07-29 10:59:34

E.R. Fleming (33 views)
2014-07-29 03:07:13

E.R. Fleming (12 views)
2014-07-29 03:06:25

pw (43 views)
2014-07-24 01:59:36

Riven (43 views)
2014-07-23 21:16:32

Riven (30 views)
2014-07-23 21:07:15

Riven (31 views)
2014-07-23 20:56:16
List of Learning Resources
by SilverTiger
2014-07-31 18:29:50

List of Learning Resources
by SilverTiger
2014-07-31 18:26:06

List of Learning Resources
by SilverTiger
2014-07-31 13:54:12

HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!