Cheshire Cat Computing

I tried version 2.18 beta2 as a test and I still get the same problem. I reverted back to version 2.15 for now since it doesn't have this issue.

I assume you are using the same RRD libraries with both, else it would be difficult to make the changeover so easily.

I'm still looking into this one but its hard when I cant duplicate the behaviour. Also, not much free time at the moment either. I'll keep you posted via this thread as things progress.

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11

On a 6 hourly summary page I have
<!-- Refresh: 60 -->
<!-- Expires: +5s -->
in the html.
On a weekly page it's :
<!-- Refresh: 900 -->
<!-- Expires: +5s -->

I've watched the traffic with a debugging proxy (fiddler), and I can see that the graphic is being requested and downloaded on every page refresh.

I've checked the expiry settings in IIS, and they weren't set at all (the default). I've been running with these settings with all previous versions of routers2. I've tested with "expire immediately" and after 1 min, and it's the same.

I've looked in the graphs folder where the images are created, and when I refresh the page I see a new image being created. The only problem is the contents do not get refreshed. So it looks like it's not a caching problem at all but something in the graphic creation process.

In that case, it definitely looks like the issue is in the recreation - either --lazy is making RRD think the graph does not need to be redrawn, or else something in the parameters passed to RRD is doing this (RRD is actually called to generate the graph every time, but due to the --lazy option it may choose not to do so).

I'll see if I can run some checks on this.

If you are a perl person, you might like to remove the --lazy in the code as this could solve the issue.

Thanks for doing these tests, it helps to narrow down the cause considerably.

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11

Ok, I'll have a go at removing the --lazy from the code.

However based on your description of --lazy I'm not wildly hopeful as I can see from the timestamps on the file that they are being recreated on every request. For example, if I delete all files from the graphs folder, and then request a new page at say 10:52 a graph will be created with a time of 10:50 and using the 10:50 data. If I request the graph again at 10:54, then a new graph is created, but the data is still from 10:50 and the timestamp in the image is 10:50. At 10:55 I'd get an updated graph, and there'd be 5 more minutes of data displayed.

I'll get back to you when I've tried the --lazy thing.

Thanks.

I suppose you are certain that you really are collecting the data at 1-min intervals, plus the RRD file itself has a 60-sec interval defined? routers2 will base the graph on the last update time if necessary (if the trailing unknown time is <5min, or if you have uselastupdate=y) and will also set the expected interval to match the reported interval from the RRD query. It is possible that this is caused by the .rrd file configuration or data collection method

You can verify this by looking at the rrdtool dump for the .rrd file in question, and/or (temporarily) removing the .rrd file and allowing MRTG to recreate it in exect concordance with the interval defined in the .cfg file, in case something caused this to mismatch.

I'm pulling at straws here, though, since much of this looks like it is specific to the way your setup works and it doesnt happen on ours. Might be due to the way you schedule data collection under windows (are you using MRTG daemon mode with a 1-min interval, or a scheduler?)

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11

I am collecting data at 1 min intervals (using cron but on windows). The start of the rrd file dump is shown at the bottom. I do get data collected for each minute in the rrd (although I've pasted a bad example).

I tried removing the --lazy (searched routers2.pl for "lazy" and commented the only line that mentioned it), and that doesn't seem to have had any affect.

I have added new targets recently and they suffer the same problem.


<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
<step> 60 </step> <!-- Seconds -->
<lastupdate> 1181753364 </lastupdate> <!-- 2007-06-13 17:49:24 CDT -->

<ds>
<name> ds0 </name>
<type> GAUGE </type>
<minimal_heartbeat> 600 </minimal_heartbeat>
<min> 0.0000000000e+000 </min>
<max> 5.0000000000e+001 </max>

<!-- PDP Status -->
<last_ds> UNKN </last_ds>
<value> 6.0000000000e+002 </value>
<unknown_sec> 0 </unknown_sec>
</ds>

<ds>
<name> ds1 </name>
<type> GAUGE </type>
<minimal_heartbeat> 600 </minimal_heartbeat>
<min> 0.0000000000e+000 </min>
<max> 5.0000000000e+001 </max>

<!-- PDP Status -->
<last_ds> UNKN </last_ds>
<value> 6.0000000000e+002 </value>
<unknown_sec> 0 </unknown_sec>
</ds>

<!-- Round Robin Archives --> <rra>
<cf> AVERAGE </cf>
<pdp_per_row> 1 </pdp_per_row> <!-- 60 seconds -->

<params>
<xff> 5.0000000000e-001 </xff>
</params>
<cdp_prep>
<ds>
<primary_value> 2.5000000000e+001 </primary_value>
<secondary_value> 2.5000000000e+001 </secondary_value>
<value> NaN </value>
<unknown_datapoints> 0 </unknown_datapoints>
</ds>
<ds>
<primary_value> 2.5000000000e+001 </primary_value>
<secondary_value> 2.5000000000e+001 </secondary_value>
<value> NaN </value>
<unknown_datapoints> 0 </unknown_datapoints>
</ds>
</cdp_prep>
<database>
<!-- 2007-06-10 23:10:00 CDT / 1181513400 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-10 23:11:00 CDT / 1181513460 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-10 23:12:00 CDT / 1181513520 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-10 23:13:00 CDT / 1181513580 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-10 23:14:00 CDT / 1181513640 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-10 23:15:00 CDT / 1181513700 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>

....

<!-- 2007-06-13 17:45:00 CDT / 1181753100 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-13 17:46:00 CDT / 1181753160 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-13 17:47:00 CDT / 1181753220 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-13 17:48:00 CDT / 1181753280 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
<!-- 2007-06-13 17:49:00 CDT / 1181753340 --> <row><v> 2.5000000000e+001 </v><v> 2.5000000000e+001 </v></row>
</database>
</rra>
... rest of xml

Thanks for this, it definitely confirms your .rrd structure and data collection to be correct (cross off another possible cause).

I take it that the 6-hour graphs are correctly showing data at 1-min granularity (IE, not all blocky) so the only issue is that we're not getting the graph to update at 5-min intervals, even if the refresh is requested manually.

We know it is not the browser caching because you say that you've seen the request go out to the server.

The graph is generated during the generation of the HTML page, and then retrieved. However we know this is being regenerated because the 'last update' time in the page is incremented (although the graph is not).

So, it looks as if the graph basetime is being forced to a 5-min boundary for some reason. This may be down to the code that avoids the trailing-unknown-race-condition problem... I'll start looking in this direction.

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11