«DiscoVeriNg HomelessNess Volume 13, Number 1 • 2011 U.S. Department of Housing and Urban Development | Office of Policy Development and Research ...»
Culture data were collected once at each organization (during January or February 2008 for ETCEH and during April or May 2009 for MICAH). Staff members completed the OSC questionnaire independently, and no supervisors were present during the testing. Organizations did not see staff members’ individual responses. Again, data from both ETCEH and MICAH were collapsed into the level-two data set.
Measurement The staff members’ use of an HMIS was measured according to the number of times each staff member logged on over the multiple-month period. Alternative measures of use that were considered include number of new clients entered, number of services provided, and number of case notes recorded. The total of logon attempts was considered the most appropriate, however, because it captured all staff members’ interactions with the HMIS. All staff members must log on to the HMIS every time they use it. The alternatives reflect job-specific HMIS interactions. For example, some
staff members do not enter new clients; they only update existing client records or run reports.
Measuring clients entered, therefore, would exclude these staff members’ use of the HMIS.
In some organizations, only a single individual used the HMIS during the year of data collection.
These individuals and their organizations were still included in the multilevel analysis, which accounts for uneven designs when higher order groups have different numbers of cases than the lower order groups (Raudenbush and Bryk, 2002). An exposure variable, the number of months that a staff member had any registered activity for the HMIS, was measured to account for the opportunity, or amount of time, that an individual had to use the system. The study included gender as a level-one predictor to control for gender differences in perceptions of the work environment (Kanter, 1977), such as stressors (Arrington, 2008; Coffey, Dugdill, and Tattersall, 2009) and job competencies (Frame et al., 2010). Unfortunately, the small sample size and limited degrees of freedom made it impossible to add more covariates to the model.
The study measured the level-two predictor—organizational culture—using the OSC questionnaire (Glisson et al., 2008).
The OSC questionnaire consists of 105 items and measures three dimensions:
(1) culture, (2) climate, and (3) work attitudes (Glisson and James, 2002). Analysis was limited to the culture scale and its corresponding subscales: (1) rigidity (14 items, alpha12 = 0.79, alpha23 = 0.74), which is the degree of order and flexibility in work habits and procedures; (2) proficiency (15 items, alpha1 = 0.86, alpha2 = 0.85), defined as the degree to which staff members are expected to be knowledgeable about and capable of providing optimal services; and (3) resistance (13 items, alpha1 = 0.79, alpha2 = 0.70), which is the ability of the environment to change work habits and procedures. Disproportionate data entry was added as a level-two covariate. Data entry among staff members was considered disproportionate if a single individual accounts for 75 percent or more of logon attempts within an organization.
Data Analysis The analysis used a two-level hierarchical generalized linear model (HGLM) (Gelman and Hill, 2007;
Raudenbush and Bryk, 2002) with a negative binomial log-link function to consider the cross-level relationship between staff members’ HMIS use and organizational culture. Although the model included three levels, only a two-level model was used, because the small number of CoCs in the third level (n = 4) made it impossible to test for variation. The negative binomial model accounted for the overdispersion (χ2 = 447.92, p =.00) in the data (Orme and Combs-Orme, 2009). In addition, the multilevel model accounted for the clustering in the data (Nair, Czaja, and Sharit, 2007;
Raudenbush and Bryk, 2002). The analysis estimated a rate of HMIS logon attempts for staff members based on the number of times they attempted to log on (the outcome variable), adjusted for the number of months they had used the system (the exposure variable). Restricted maximum likelihood estimation was used rather than full maximum likelihood estimation, because the former is considered less biased than the latter with small samples (Nair, Czaja, and Sharit, 2007). A test of the null model, including only the outcome and exposure variables, indicated that random variation existed among organizations in frequency of HMIS logon attempts (χ2 = 89.93, p =.00).
Refers to the ETCEH sample.
Refers to the MICAH sample.
The full model included (1) number of months of use (exposure variable), (2) proficiency and rigidity at level two,4 (3) gender at level one, and (4) the cross-level interactions by gender (that is proficiencyXgender and rigidityXgender). The interaction between rigidity and gender was not statistically significant and did not improve the model fit. Consequently, it was not included in the final model. The full model is specified as shown in equation (1) below.
ηij = γ00 + γ01(dd) + γ02(proficiency) + γ03(rigidity) + γ10(gender) + γ11(proficiencyXgender) + μ0j + rij (1) where ηij is the log of the monthly rate of HMIS logon attempts for staff member i in organization j.
γ00 is the average rate of client entries for a staff member. γ01(dd) is the difference in HMIS logon attempts between organizations with a disproportionate data entry system and those without.
γ02(proficiency) is the 1-point change in HMIS entry for every 1-point increase in organizational proficiency. γ03(rigidity) is the 1-point change in HMIS entry for every 1-point increase in organizational rigidity. γ10(gender) is the difference in logon attempts for males and females.
γ11(proficiencyXgender) is the 1-point change in the rate of HMIS logon attempts as a function of the interaction between organizational proficiency and gender. μ0 is the random variation among organizations, and rij is the random variation among staff members.
Results HMIS use was measured at the individual level and results are reported in univariate form by individuals, in the aggregate form by organizations, and in the bivariate form by looking at the relationship between the concept and continuum-of-care (CoC) membership. Results from the multivariate analysis follow the bivariate results. Two of the organizations surveyed did not use HMIS during the year of data collection. For this reason, they were excluded from the multilevel model.
Univariate Exhibit 1 shows individual use of the HMIS, as measured by the number of times a staff member logged on to the system during the time period. The kurtosis value for use indicates a strong positive skew in the data, so medians are interpreted rather than means. Usage ranged from 2 to 719, with a median of 47.5 times. These results suggest that most staff members did not log on frequently, but a small percentage of the users were outliers who logged on far more than the others. Months using the system show a more normal distribution, with a range from 1 to 10 and a median of 9 months for CoC 2-4, and with a range of 1 to 12 and a median of 8 months for CoC 2-4.
Results at the aggregate organizational level, also shown in exhibit 1, suggest that a wide variation exists in how frequently the organizations use the HMIS. Total logon attempts by staff members ranged from 5 to 3,688 (M = 660.92, s.d. = 952.1) per organization. The maximum time that a staff member at an organization had used the system ranged from 1 to 12 months (M = 9.33, s.d. = 3.36).
The mean number of staff members using the HMIS at an organization was 8 (s.d. = 9.93), ranging from 1 to 35 users. The mode is one, however, suggesting that many of the organizations only have Resistance was not included in the final model because of its high correlation with rigidity (r = 0.603, p 0.001).
Bivariate Individual comparisons of HMIS logon attempts, across CoC, service provider type, and gender also showed variability. The total number of times that staff members in each CoC attempted to log on to the system ranged from 616 for CoC 1 to 6,106 times for CoC 4. This distribution in logon attempts is reflected in the distribution of HMIS users. CoC 1 accounted for 9.15 percent of the users and CoC 4 accounted for 35.9 percent.
The relationship between the number of HMIS logon attempts and the type of service provider, showed similar disparities. The mean number of logon attempts ranged from 28 for staff members in emergency shelters to 90 in transitional housing. Men reported a higher level of use with a median number of logon attempts equaling 66 compared with 46 for women.
Results continued to suggest variability at the organizational level. Exhibit 2 displays comparisons of HMIS use, aggregated to the organizational level, and compared across CoC. Aggregated HMIS logon attempts ranged from a median of 33 for organizations in CoC 2 to 220 in CoC 4. Also, the number of clients entered into the HMIS ranges from an organizational mean of 608.33 (s.d. = 417.63) for CoC 4 to 1,038.07 (s.d. = 578.02) for CoC 3. Almost no variation existed in the number of months that staff members in each of the CoC used the system, with means ranging between 6.53 (s.d. = 3.48) and 6.74 (s.d. = 3.77).
Multilevel Model Hypothesis One Main Organizational Effects. Exhibit 3 reports the results of the multilevel model. The model did not support the hypothesis that culture characteristics affected HMIS use. When controlling for the other variables in the model, rigidity was not statistically significant (B = – 0.036, ERR = 0.964 (0.939, 0.991), p =.011). Similarly, when controlling for the other variables in the model, proficiency was not statistically significant.
Hypothesis Two Interaction Effects. Results supported the second hypothesis that an interaction between organizational-level and individual-level characteristics would affect HMIS use. The interaction between proficiency and gender (B =.033, ERR = 1.085, p =.016) was statistically significant.
Because proficiency is a T-score,5 the event rate ratio (ERR) lacks intrinsic meaning. The ERR, which is the unstandardized beta coefficient exponentiated, quantifies the strength and direction of the relationship between independent and dependent variables. To facilitate interpretation, the ERR was transformed by multiplying the coefficient by 10 and exponentiating the value: exp(0.033*10).
Results of this calculation on the ERR for the proficiencyXgender interaction indicate that for every one standard deviation increase (10 points) in organizational proficiency, the rate of logon attempts for men increases by a factor of 1.391 (39 percent). They are more likely to use the HMIS in organizations with higher levels of proficiency.
Discussion The most important finding in the current study is that the effect on HMIS log attempts of an organizational-level variable—proficiency—is moderated by gender, an individual-level characteristic. This finding confirms research showing that the interaction between individual and organizational attributes can affect service provision (North et al., 2005). The present study showed no effects of organizational culture on women, but men were more likely to attempt to log on in organizations that valued proficiency. Possible explanations for the gender differential include differences in job status and responsibilities. For instance, men may be more likely to hold positions of authority. The individuals in authority and who hold leadership positions are often those responsible for developing policies and procedures, and leadership is partly responsible for creating and maintaining the organizational culture (Schein, 1992). If men are holding leadership positions, they may be largely involved in the shaping of a culture that values innovation and competency. In this study, however, men and women were relatively equally likely to hold authority positions (23.3 and 25.8 percent, respectively).
Alternatively, women may be affected similarly, but this relationship was not observed due to limited statistical power. The pilot study showed, however, that gender acted as a moderator on the effect of organizational culture for both men and women. It is also important to note that the effects for men did not become apparent until organizations reached very high levels of proficiency. Further research is necessary to completely understand the interactions between gender and organizational culture.
Unlike in the pilot study (Cronley and Patterson, 2010), the present study did not show a relationship between rigidity and HMIS use, perhaps because the studies used different outcome measures.
The pilot study examined HMIS use as measured by the number of new clients a staff member entered into the system. The current study examined HMIS as measured by the number of times T-scores differ from the T-test values reported in Exhibit 3. The T-test values are based on Student’s T distribution and indicate how far the sample deviates from this distribution. The T-scores discussed here are the standardized values of the organization’s culture scores (based on rigidity, proficiency, and resistance). T-scores have a mean value of 50 and standard deviation of 10, thus, a score of 60 indicates that the value is one standard deviation above the mean.
a staff member logged on to the system (for reasons discussed earlier in Measures and subsequently in Limitations). It may be that the entry new clients was a better indicator of HMIS use. Logon attempts serve as a skeletal indicator of HMIS use—an indicator that does not portray substantive interaction with the system. Because the primary purpose of using an HMIS is to collect data about the homeless, records of new client entries may offer a clearer picture of the system’s use.
The lack of statistical significance in the current study may be related to limited use of the system as well. The study suggested that sampled organizations are not using the HMIS to its full capacity.