Based on the CIMISS (China Integrated Meteorological Information Service System) database and using consistency ratio (in depth test of frozen soil using absolute value of error) as the inspection index, the data of 15 automatic comprehensive judgement items in records of national meteorological stations in Shandong are compared with the synchronous observation results of the stations, and assessment analysis is carried out. The results are shown below. 1) The consistency ratio of the judgement of total cloud cover is 65.8％ and the absolute value of cloud cover deviation being 1 accounts for 51.1％. Most of the inconsistency occurs when the observation is between overcast sky and full clouds but the judgement results are cloudless or partly cloudy. The overall consistency ratio of cloud height after classification is 85.0％ compared with observations. The consistency ratio of cloud height is higher than that of cloud cover at each station. 2) The consistency ratio is above 92.0％ between observations and comprehensive judgement results of 9 weather phenomena, including rime, glaze, fog, mist, suspended dust, blowing sand, sand and dust storm, snow cover, and icing; the consistency ratio of frost, haze, and dew is above 75.0％. The hit rate of judging icing, mist, fog, and frost is high; the omission rate of judging snow cover, glaze, and blowing sand is high; the vacancy rate of judging sand and dust storm, glaze, blowing sand, and rime is high. 3) The judgement effect of glaze and rime is affected by the sample size of a single station; there is no observation task of grass temperature for the national meteorological observation stations during the assessment period, which is the main reason for the high omission rate of judging snow cover. When there is precipitation, the concurrent phenomena of visual range obstruction cannot be judged, and this explains the high omission rate. 4) The average absolute error of frozen soil data during the assessment period is 1.6 cm, the absolute value of error between judgement and observation from 1 cm to 10 cm accounts for 82.8％, and the judgement performs well. Through the assessment analysis, it can be seen the consistency is higher between comprehensive judgement results and observations based on multi-source data of satellite, sounding, automatic meteorological station, radar, and lightning location system combined with reanalysis products of models, which can better meet the needs of weather forecast.